Can we generate locale keys in record form? about easy_localization HOT 12 CLOSED

I've made something similar and wanted to propose it to be added to this repo, but couldn't find the time for a proper PR.

The high level overview is:
You have a watcher assigned to your localization file. Whenever the file changes it triggers a dart script that generates locale strings in a convenient format inside your project. Basically it turns your json into a set of classes and json keys into its properties. The code autocompletes so there's no need to write raw strings hence there's no human error possible. Also the scripts executes instantly on file change, so there's no need to run the code generation tool after every edit. There's a keyPath property assigned to every key for pluralization support.

Example localization file
en.json

{
    "example": {
        "title": "This is a title",
        "numberOfExamples": {
            "one": "{} example",
            "two": "{} examples",
            "few": "{} examples",
            "other": "{} examples"
        },
        "map": {
            "title": "Map title",
            "mapInsideMap": {
                "title": "Map inside map title"
            }
        }
    }
}

Example output file
locale_keys.g.dart

abstract class Loc {
  static var example = _ExampleKeys();
}

class _ExampleKeys {
  String title = "example.title";
  var numberOfExamples = _ExampleNumberOfExamplesKeys();
  var map = _ExampleMapKeys();
  String get keyPath => 'example';
}

class _ExampleNumberOfExamplesKeys {
  String one = "example.numberOfExamples.one";
  String two = "example.numberOfExamples.two";
  String few = "example.numberOfExamples.few";
  String other = "example.numberOfExamples.other";
  String get keyPath => 'example.numberOfExamples';
}

class _ExampleMapKeys {
  String title = "example.map.title";
  var mapInsideMap = _ExampleMapMapInsideMapKeys();
  String get keyPath => 'example.map';
}

class _ExampleMapMapInsideMapKeys {
  String title = "example.map.mapInsideMap.title";
  String get keyPath => 'example.map.mapInsideMap';
}

Example usage.

import '../../../generated/locale_keys.g.dart';

Text(Loc.example.title.tr());
Text(Loc.example.map.title.tr());
Text(Loc.example.map.mapInsideMap.title.tr());
Text(plural(Loc.example.numberOfExamples.keyPath, 5));

To use this.

1. Add watcher to your dev_dependencies in pubspec.yaml

dev_dependencies:
  watcher: ^1.1.0

2. Create loc_generate.dart in the root of your project (same depth as pubspec.yaml is at). Copy the following into it. Change translationAssetPath to the path of one of your translation files that you are going to be updating first.

import 'dart:convert';
import 'dart:io';

import 'package:watcher/watcher.dart';

String capitalize(String s) => s[0].toUpperCase() + s.substring(1);

const translationAssetPath = 'assets/translations/en.json';

void generateProperties(
  StringBuffer buffer,
  Map<String, dynamic> jsonData,
  List<String> paths,
  int depth,
) {
  List<String> classes = [];
  var prefix = depth > 1 ? '${paths.join('.')}.' : '';
  for (var key in jsonData.keys) {
    var value = jsonData[key]!;
    if (value is Map) {
      classes.add(key);
      String className =
          '_${[...paths, key].map((e) => capitalize(e)).join()}Keys';
      if (depth == 1) {
        buffer.writeln('  static var $key = $className();');
      } else {
        buffer.writeln('  var $key = $className();');
      }
    } else {
      if (depth == 1) {
        buffer.writeln('  static const String $key = "$prefix$key";');
      } else {
        buffer.writeln('  String $key = "$prefix$key";');
      }
    }
  }

  if (prefix.isNotEmpty) {
    buffer.writeln(
        '  String get keyPath => \'${prefix.substring(0, prefix.length - 1)}\';');
  }

  buffer.writeln('}');
  buffer.writeln('');

  for (var key in classes) {
    var newPaths = [...paths, key];
    String className =
        '_${[...paths, key].map((e) => capitalize(e)).join()}Keys';
    buffer.writeln('class $className {');
    generateProperties(buffer, jsonData[key], newPaths, depth + 1);
  }
}

void generate() {
  final File jsonFile = File(translationAssetPath);

  final String jsonString = jsonFile.readAsStringSync();
  final Map<String, dynamic> jsonData = json.decode(jsonString);

  final StringBuffer buffer = StringBuffer();

  buffer.write(
    'export \'package:easy_localization/easy_localization.dart\';\n\n',
  );

  buffer.writeln('abstract class Loc {');
  generateProperties(buffer, jsonData, [], 1);

  final generatedDirectory = Directory('lib/generated');
  if (!generatedDirectory.existsSync()) {
    generatedDirectory.createSync(recursive: true);
  }

  const outputFilePath = 'lib/generated/locale_keys.g.dart';
  File(outputFilePath).writeAsStringSync(buffer.toString());
}

void main() {
  final watcher = FileWatcher(translationAssetPath);

  watcher.events.listen((event) {
    if (event.type == ChangeType.MODIFY) {
      // ignore: avoid_print
      print('File changed: ${event.path}');
      try {
        generate();
      } catch (e) {
        // ignore: avoid_print
        print('aborting $e');
      }
    }
  });
}

3. Open a separate terminal window and start the script.
   dart loc_generate.dart

In conclusion. This approach is very convenient but has a few issues.

• It only creates keys based on one file, so keys between your localization files must be synchronized
• Currently the localization file path is hardcoded into the script
• It has a dependency on watcher, but I really think it's no big deal

from easy_localization.

Right, so records is basically tuples. I've been using them without even realizing it, here's a line from my project (String, String) _getTitleAndDescription(). :)

I think both records and class approaches deserve to exist and personally I wouldn't spend time to redo something that is already working to have an output with a different format, considering we're not spending any time within the generated file itself and that they solve the problem almost identically. If it sounds like a fun challenge to you, I'd love to see the records generator you create.

from easy_localization.

@tirendus Thank you very much for the example. I have almost finished the implementation in my application. For now I was manually generating the keys. I was going to prepare a generator when I had the opportunity. Obviously I wanted to try and see.

Now I'm pretty sure that this approach is much more useful than generating string variables.

With the renaming feature I can change the names of the master keys. At the moment the Dart plugin doesn't allow renaming named parameters, but I think that will come in the future.

I'm not sure why we need to create keys as classes. Records is very convenient for this. Creating them as records seems simpler and more readable to me.

I wonder what you think about this.
Which one would be more useful and performant?

from easy_localization.

Hello @hasimyerlikaya

I've created them as classes to allow inner child nodes and frankly I didn't even know records existed until today, it's like tuples? This approach allows for any depth of inner translation dictionaries which is useful in my case, where I have around 500 translated strings that are grouped by tabs/pages/sections within pages etc. It also allows me to add actual methods to the classes, so it's very versatile.

The performance hit with using classes versus records is negligible, I wouldn't worry about it. Regarding the readability, your approach is much closer to the original json file, which I find very readable, but I never look into the file with generated strings, I know the key names by how I name them in the json file itself. For example if I create a file

en.json

{
    "common": {
        "validators": {
            "numberTooLong": "Numbers must be 7 characters or less"
        },
        "errors": {
            "noInternet": "No internet access"
        }
    }
}

I can access numberTooLong message by Loc.common.validators.numberTooLong.tr(). Maybe I should actually encapsulate the tr() logic as well when time permits :)

from easy_localization.

Hello @tirendus

With Dart 3.0, it was introduced to create an anonymous data structure without the need to create a class. Actually, I think this structure was added for exactly these situations.

We can also use Records if we want a function to return more than one value. Previously, it was necessary to create a class, use a list or map for this. Now there is no need for this.

I think it is a very useful structure.

For detailed information, see: https://dart.dev/language/records

As a result, there is no difference in our access to the keys. Since both are nested structures, there is no difference in grouping properties and access.

I didn't need any method in the key file. Honestly, I wonder what kind of method you are adding. 🤔

I agree about reading the generated key file, we won't need much. Except for renaming a key group sometimes, we probably won't need it. Since I am currently creating keys manually, reading is a bit of a distraction. But if I use a generator, I probably never look at it :)

I am thinking of editing the generator you gave in the previous message and turning it into records. It would be good to have two alternatives. Those who need it can use it too.

Json:

    "incomeTransaction": {
        "fields": {
            "amount": "Amount",
            "isPaid": "Status",
            "paymentDate": "Payment Date",
            "transactionDate": "Transaction Date",
            "transactionTag": "Tag",
            "remainingTime": "Remaining Time",
            "note": "Notes",
            "paymentCount": "Payment Count"
        },
        "actions": {
            "update": "Edit Payment",
            "detail": "Payment Detail",
            "addTag": "Add Tag"
        },
        "validations": {
            "amount": {
                "notNull": "You must enter the amount",
                "lessThen": "Amount cannot be greater than 1,000,000,000"
            },
            "paymentDate": {
                "notNull": "You must enter the payment date"
            },
            "isPaid": {
                "notNull": "You must select the transaction status"
            },
            "firstPaymentDate": {
                "notNull": "You must enter the start date"
            },
            "paymentCount": {
                "notNull": "You must enter the payment count",
                "lessThanOrEqual": "Payment count cannot be greater than 100"
            }
        },
        "hints": {
            "paymentCount": "Maximum 100"
        }
    },

Keys:

static const incomeTransaction = (
   fields: (
     amount: "incomeTransaction.fields.amount",
     isPaid: "incomeTransaction.fields.isPaid",
     paymentDate: "incomeTransaction.fields.paymentDate",
     transactionDate: "incomeTransaction.fields.transactionDate",
     transactionTag: "incomeTransaction.fields.transactionTag",
     remainingTime: "incomeTransaction.fields.remainingTime",
     note: "incomeTransaction.fields.note",
     paymentCount: "incomeTransaction.fields.paymentCount",
   ),
   actions: (
     update: "incomeTransaction.actions.update",
     detail: "incomeTransaction.actions.detail",
     addTag: "incomeTransaction.actions.addTag",
   ),
   validations: (
     amount: (
       notNull: "incomeTransaction.validations.amount.notNull",
       lessThen: "incomeTransaction.validations.amount.lessThen",
     ),
     paymentDate: (notNull: "incomeTransaction.validations.paymentDate.notNull",),
     isPaid: (notNull: "incomeTransaction.validations.isPaid.notNull"),
     firstPaymentDate: (notNull: "incomeTransaction.validations.firstPaymentDate"),
     paymentCount: (
       notNull: "incomeTransaction.validations.paymentCount.notNull",
       lessThanOrEqual: "incomeTransaction.validations.paymentCount.lessThanOrEqual",
     ),
   ),
   hints: (paymentCount: "incomeTransaction.hints.paymentCount",)
 );

There is no problem using too many nested keys. Right now my deepest key is
AppLang.incomeTransaction.validations.paymentCount.lessThanOrEqual

The fact that the keys look like a json file attracts me more 😅

from easy_localization.

Today I started working on a new package.
I have a good idea.

I'm thinking to combine the key and language files and cover all needs in a single file.

We can load languages using AssetLoader. The process of creating the main body is finished. Now it's time to fill it. I will let you know when it is finished :)

from easy_localization.

Hello @tirendus,

I want to ask you something.

I think it might be good combine keys and text.
What do you think about this structure?
Is that a good idea?

@AppLangSource()
class AppLanguage {
  static const incomeTransaction = (
    fields: (
      amount: (
        tr: "Fiyat",
        en: "Amount",
      ),
      paymentDate: (
        tr: "Ödeme Tarihi",
        en: "Payment Date",
      ),
    ),
    actions: (
      update: (
        tr: "Düzenle",
        en: "Edit",
      ),
      detail: (
        tr: "Detay",
        en: "Detail",
      ),
      addTag: (
        tr: "Etiket Ekle",
        en: "Add Tag",
      ),
    ),
    validations: (
      amount: (
        notNull: (
          tr: "Tutar girmelisiniz",
          en: "You must enter an amount",
        ),
        lessThen: (
          tr: "Tutar 100'den küçük olmalı",
          en: "Amount must be less than 100",
        ),
      ),
    ),
  );
}

from easy_localization.

I think it's best to stick to loading only 1 file into memory, as certain users might have a lot of localizations and you don't want to keep all of them in the memory. The memory footprint will probably be negligible, but it's still a good idea to use as little resources as possible. You also have to give user easy access to values of those keys depending on the currently selected locale. For instance, it would be a regression to force user to check their locale and pick an appropriate localization key using full path like validators.amount.notNull.en.

from easy_localization.

Actually, I was a bit incomplete.
The AppLanguage class is my source file where I define the keys and texts.

I will read this file and generate the AssetLoader file of the easy_localization package. I am currently writing a build_runner package for this.

CodegenLoader is ready, I'm just looking for a way to read and parsing my definition file. I found it, but it's not quite at the level I want yet.

To summarize, I want to have a single dart file "AppLanguage" instead of keeping the languages in two separate json files and creating a key file on top of it. Because we define the keys in 3 different places. And the texts are in separate files. Now they will be found one under the other.

Source:

@AppLangSource()
class AppLanguage {
  static const incomeTransaction = (
    fields: (
      amount: (
        tr: "Fiyat",
        en: "Amount",
      ),
      paymentDate: (
        tr: "Ödeme Tarihi",
        en: "Payment Date",
      ),
    ),
    actions: (
      update: (
        tr: "Düzenle",
        en: "Edit",
      ),
      detail: (
        tr: "Detay",
        en: "Detail",
      ),
      addTag: (
        tr: "Etiket Ekle",
        en: "Add Tag",
      ),
    ),
    validations: (
      amount: (
        notNull: (
          tr: "Tutar girmelisiniz",
          en: "You must enter an amount",
        ),
        lessThen: (
          tr: "Tutar 100'den küçük olmalı",
          en: "Amount must be less than 100",
        ),
      ),
    ),
  );
}

Result:

from easy_localization.

That would work for your case, but multiple files and json source files is the correct approach. Couple of reasons why:

1. json is universal, dart records will only work for dart projects. I recently had to rewrite a project from react-native to dart, I just had to copy json translation files from one project into another. I'd have to rewrite those files from scratch if they were in records form.
2. The translation files are often given to a translation agency and then copywriters. Inside the translation agency appropriate language files are given to workers that know the source and destination languages. If you only have one file, the agency will ask you to split it on a per language basis so they can distribute files to translate to appropriate agents.
3. The more translation languages you have, the harder it will be to work with this file
   If you want your package to be scalable, make it modular and use universal solutions.

from easy_localization.

I agree with all your points. You are absolutely right on all of them.
The reason why I prefer such an approach is that it will be useful for renaming and grouping keys.

When we work with json, it's a lot more work to rename the keys. The keys have to be exactly the same in multiple files. For example, in my project, I first created the texts without grouping the keys, then I decided to group them and now I need to reorganize all the keys.

If I worked with Dart, this would be very easy. I would just copy the keys I want to the places I want and recreate the language files.

As for the disadvantages;
1 - Json is universal, I can read the json file and create the dart file. This makes it easier to migrate from other projects. I can add import feature to the package.

2- In the Loader class I created for AssetLoader, the languages are already separate and in json format. I can save the data there to a file and give it whenever I want. I can even add this to the package as a feature. There will be an export feature.

It sounds like a bit of an unnecessary approach, but I don't like working with Json :D
I also learned a lot of good things with two days of work. I gained great experience :)

from easy_localization.

Hi @tirendus,

I have published my package.
It supports all your needs.
You may want to look at it.

https://github.com/EFA-dev/app_lang

from easy_localization.