Giter Club home page Giter Club logo

dyml's Introduction

Dyml

The YAML and DYML processor of your dreams, with ultimate comment support, written from scratch in Java!

Features

  • Amazing support for side, multiline, regular comments and line breaks.
  • Written from scratch with performance and usability in mind.
  • Only key-features of YAML implemented, thus very small, blazing fast and easy to use.
  • API design, core ideas:
    • Rely on default values
    • Either null or nothing (no empty values/strings)
    • Keep it simple but powerful (few objects that can do a lot)

example.yml code

important: Everything else that is not explicitly mentioned in this file is not supported

# Comments and
# multiline comments support.
supports-lists: 
  - Hello World!
  - 2nd value
supports-hyphen-separation: awesome! 
or separation by spaces: great! # side-comments supported!
and.dots.like.this: wow!
or:even:colons: puh!

# Complex hierarchies supported.
g0:
  g1a:
    g2a: wow!
    g2b: <3
  g1b:
    - v1 # side-comments in lists
    # This is also a side-comment, for the value below
    - v2

example.dyml code

important Everything else that is not explicitly mentioned in this file is not supported
 Comments and
 multiline comments support.
key value
 Complex hierarchies supported.
g0
  g1a
    g2a wow!
    g2b <3
  g1b great!

Read more about the .dyml file type here. Note that the API for .dyml files is in beta and not all features of the .yaml API were ported over yet.

Some extras:

  • DirWatcher | Directory watcher with recursive directory watching support.
  • YamlDatabase | (BETA) Yaml based, lightning fast database.

Examples

All examples can be found as tests here.

Show frequent YAML mistakes

It's fine to have colons in keys, as long as there is no space after it. Here is a small quiz, determine the key and value for the following yaml section:

hello:there: my : friend: !

Answer: The key is hello:there and the value my : friend: !.

Benchmarks

Dream-Yaml seems to be about 9x faster than SnakeYAML , 8x faster than YamlBeans , 4x faster than eo-yaml and 3x faster than Simple-Yaml.

Open/close details

FAQ

Difference between 'loaded' and 'added' modules? The only difference, is that loaded modules cannot have default values set. They are basically the raw output from your yaml file. In-Edit modules get created when you call the add() method. Their initial value is taken from the loaded module with the same keys.
How are null/empty values handled?
parent:
  key1:               # this value is null
  key2: ~             # not null, but a string
  key3: null          # not null, but a string
  key5: "null"        # not null, but a string
  key5: ""            # this value is null (note that if you disable the remove quotes post-processing option, this is a string("") and not empty, otherwise this gets turned into a null value)
To sum it up: Empty values do NOT exist. Null values exist. Note that null values are removed from the modules values list, in the post-processing part while parsing the yaml file. You can disable it though, if you want.
Fallback to default values? When the 'real value' is null, return the default value. This feature is enabled by default. You can change it for each individual module.

dyml's People

Contributors

nelmindev avatar osiris-team avatar rubenicos avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

rubenicos

dyml's Issues

Hierarchies wrong when adding a new module!

Easiest should be by showing the issue in an example.
Lets imagine we load the following yaml file:

g0:
  g1:

Now we want to add a new module with these keys: [g0, g1, g2].
Because of this in the createUnifiedList() method:

if (inEditModule.getKeys().size() > 1) {
                DYModule parent = new UtilsDYModule().getExisting(inEditModule.getKeys().subList(0, inEditModule.getKeys().size() - 1), loadedModules);
...

we get the following output after adding the new module and saving:

g0:
  g1:
g0:
  g1:
    g2:

instead of:

g0:
  g1:
    g2:

List unsupported error in setDefValues

public List<String> get(String path, String[] defaultValue, int topLineBreaks, String... comments) throws NotLoadedException, IllegalKeyException {
        YamlSection yamlSection;
        if (isPathSplitRequired(path)) {
            yamlSection = this.config.put(pathSplit(path)).setDefValues(defaultValue).setCountTopLineBreaks(topLineBreaks).setComments(comments);
        } else {
            yamlSection = this.config.put(path).setDefValues(defaultValue).setCountTopLineBreaks(topLineBreaks).setComments(comments);
        }
        System.out.println(yamlSection.asStringList().toString());

        return isValueNull(yamlSection) ? List.of(defaultValue) : yamlSection.asStringList();
    }

I'm using the same code as above. If I put parameter(defaultValue) in setDefValues, it does not work properly(Unconditional initialization, asStringList() return empty list etc...). Can you check about this situation? (If I enter a value directly instead of the defaultValue parameter, it works properly, The defaultValue parameter is new String[]{"foo", "bar"})

EDIT : The "List<SmartString>" parameter is not working properly either.

Drop support for side-comments (yaml)

Drop support for side-comments to increase performance and fix some bugs that otherwise would require more complicated solutions.
Also nobody uses really side-comments anyways.

Modules of the same generation get ordered wrong

    @Test
    void add() throws IOException, DuplicateKeyException, DYReaderException, IllegalListException, DYWriterException, NotLoadedException, IllegalKeyException {
        DreamYaml yaml = new DreamYaml(System.getProperty("user.dir") + "/src/test/tests.yml");
        yaml.reset();
        DYModule m1 = yaml.add("test-put").setValues("value");
        DYModule m2 = yaml.add("test-put", "c1").setValues("value");
        yaml.saveAndLoad();

        assertNotNull(m1);
        assertNotNull(m2);

        assertNotNull(m1.asString());
        assertNotNull(m2.asString());

        // Test putting/adding a new module to the file
        yaml.add("test-put-new").setValues("wow!").setComments("This is a comment!");
        yaml.saveAndLoad();
        yaml.printFile();

        // Test putting/adding a new module to the file (in a hierarchy)
        yaml.add("test-put", "c1", "c2").setComments("This is a comment!").setValues("value!");
        yaml.saveAndLoad();
        yaml.printFile();

        // Test putting/adding a new module to the file (in a hierarchy with multiple modules in the same generation)
        yaml.add("test-put", "c1-1").setComments("This is a comment!").setValues("value!");
        yaml.add("test-put", "c1-2").setComments("This is a comment!").setValues("value!");
        yaml.add("test-put", "c1-3").setComments("This is a comment!").setValues("value!");
        yaml.saveAndLoad();
        yaml.add("test-put", "c1-4").setComments("This is a comment!").setValues("value!");
        yaml.saveAndLoad();
        yaml.printAll();
        yaml.printFile();
    }

Initialize DreamYaml with InputStream

Example:

ClassLoader classLoader = getClass().getClassLoader();

DreamYaml yaml = new DreamYaml(classLoader.getResourceAsStream("somePath/yamlStuff.yml"));

With a little change on DYReader:

public void parse(DreamYaml yaml) {
    ...
    if (yaml.getFile() != null) {
        // Some stuff if file path is present
        ...
        reader = new BufferedReader(new FileReader(file));
    } else {
        // Use InputStream instead
        reader = new BufferedReader(new InputStreamReader(yaml.getStream()));
    }
}

UTF-8 not available?

There is a problem that certain characters are breaking. Can't I save the file as UTF-8?

How to handle \ ?

Dyml behaves weird if \ are inside it. Example:

windows-path D:\Coding\C\LJDB

Add line spacing

Methods:

module.getTopLineSpacing() // Returns int representing the amount of empty lines to the next upper module
module.setTopLineSpacing() 

module.getBottomLineSpacing() // Returns int representing the amount of empty lines to the next lower module
module.setBottomLineSpacing()

Example usage:

m1: val # has 0 top line spacings and 0 bottom line spacings
m2: val # has 0 top line spacings and 2 bottom line spacings 


m3: val # has 2 top line spacings and 0 bottom line spacings 

Lets imagine we set the 'm2' modules bottom line spacings to 4?
Then what happens to the "m3" module?
Edit we could remove setBottomLineSpacing() and only use setTopLineSpacing() instead, to prevent that inconsitency between modules.

What about comments?
Since the line spacing is between modules (a comment is part of a module) the comment wouldn't have line spacings to its key.

Nullpointer

[20-11-2021 21:03:25][AP][WARN] Type: java.lang.NullPointerException
[20-11-2021 21:03:25][AP][WARN] Stacktrace:
[20-11-2021 21:03:25][AP][WARN] com.osiris.dyml.DYModule.setDefComments(DYModule.java:514)
[20-11-2021 21:03:25][AP][WARN] com.osiris.dyml.DYModule.setDefComments(DYModule.java:508)

Add DB like behavior

A single YAML file could be divided into tables.
Example:

tables:
  - table1Name:
      - column1:
         - "This value is in column1" # First row
         - "Another value"
         - 
      - column2:
         -  "This value is in colum2"   
         -      # This means there is no value
         -  "" # This means the value is empty

  # A tastier example
  - favorite-food:
     food, likes:
       - 1: "Bread" "1240" 
       - 2: "Milk" "889"  

Hierachies not being read correctly

Suppose I have the following yaml:

something: true
alot_of_things:
  athing: "something is not right"
  anotherthing: "nah its right"

When I save it, it becomes this:

something: true
something:
  athing: "something is not right"
  anotherthing: "nah its right"
alot_of_things:
  athing:
  anotherthing:

Any idea on why this is happening?

The colon(':') in the value of the list data type causes an error.

In a single String data type, the inclusion of colon in the value does not cause errors,
but in the List data type, errors occur.

ex)

TestKey1: "TestValue1: abc"
TestListKey1:
  TestListKey2: "defaultTestValue2"

TestKey1 works fine without any problems, but TestListKey2 does not work properly.

Rewrite Yaml classes (remove list based design)

So that something like this is possible:

key1:
  child1:
    child2: value
Yaml child1 = yaml.get("key1", "child1");
child1.get("child2").asString();

This would require a tree-based design (linked list), where parent objects contain children.

Comments behave weird if they get removed

Example:

First run:

yaml.put("hello").setComment("my comment");
yaml.save();

First run output:

# my comment
hello:

Then in the second run, I change the code to this:

yaml.put("hello");
yaml.save();

Second run output is something like this:

#                                               my comment
hello:

Enhance read/write speed even further

image
To run this benchmark yourself simply clone this repo and run this test: https://github.com/Osiris-Team/Dream-Yaml/blob/d06e9d1a4985c985f9b3f589875d818938e2bb8d/src/test/java/YamlBenchmarks.java#L46

Research (read)

  • The postprocessing costs around 0,050ms - 0,100ms of extra time
  • The debugger costs around 0,100ms of extra time
  • Currently, we are using .readLine() and then go through that line again and check it char by char. Instead of going through it twice maybe there is a way of going through it only once.

Research (write)

The java.nio.channels package, specifically the FileChannel class provides methods for reading/writing stuff at specific positions of the file. This may drastically increase the writing speed for bigger files, since we would only write the changes instead of the whole file.
Possible logic for the above:

  1. Check if the file is empty. If it is simply parse and write all modules to it. Otherwise, continue:
  2. During read attach the keys and values start/end positions to the module. Now if the module was modified, aka the value/key was changed we write to those specific positions only.

There are however some negative aspects to this approach:

  • Extra memory needed to store each keys/values positions
  • The more time passes between read and write, the more likely it is that the file was changed in between and thus the positions are not accurate anymore which could lead to breaking the yaml file
  • We would need to recalculate the position of all keys/values with each modification I think
  • The above would not work for Input-/Output- Streams
  • We would also need to store the old/unmodified value/key to recalculate the location

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.