julioz / floorplan Goto Github PK

So far, dbml-parser doesn't properly handle column defaults defined in DBML, as such:

Table users {
    id integer [primary key]
    username varchar(255) [not null, unique]
    full_name varchar(255) [not null]
    gender varchar(1) [default: 'm']
    created_at timestamp [default: `now()`]
    rating integer [default: 10]
}

You can set default value as:

number value starts blank: default: 123 or default: 123.456
string value starts with single quotes: default: 'some string value'
expression value is wrapped with parenthesis: default: now() - interval '5 days'``
boolean (true/false/null): default: false or `default: null`

I used to use floorplan-cli through Github Actions which worked well (thanks for your amazing work).

      - name: Install floorplan
        run: brew install julioz/tap/floorplan

For some months now, it's not working anymore, I get this error (also in local):

prompt ~ brew install julioz/tap/floorplan   
Error: julioz/tap/floorplan: wrong number of arguments (given 1, expected 0)

Does something change about the brew tap ? Does I have still the good path to floorplan ? I did not see any changes on the documentation

Thanks for your help

This would be cool as a Gradle Plugin to automatically find the latest Room schema files instead of having to pass their file path.

Instead of relying on the field note feature in DBML, we can also allow for static rendering of the ER diagrams in combination with nullable types by appending a marker to the type definition.

According to DBML's schema, we can append a (?) text suffix to the data types when they are nullable.

So, for example, when a command line argument is passed, FloorPlan can render

varchar(?)

for a nullable TEXT Room field.

Enforced by this (currently ignored) test.

Instead of forcing a full block like for the output format, like

floorPlan {
  outputFormat {
    svg { 
      enabled = true
    }
  }
}

We could allow for

floorPlan {
  outputFormat {
    svg.enabled = value
  }
}

or even simpler:

floorPlan {
  outputFormat.svg.enabled = value
}

With the multi-output support planned for #45, running the CLI for FloorPlan with

./gradlew run --args="../samples/tivi-26.json --format=dbml,svg,png,dot --output=multi-test/a"

becomes cumbersome, since we are forced to specify a file (and its name) through the --output argument. We should change that to be in par with the Gradle Plugin, and instead expect a directory for the output. We can then reuse the input file name as the output files naming.

Allow for a configuration on diagram rendering for usage of crow-foot notation.

digraph {
    rankdir=LR
    node [shape=point style=invis]

    f -> F [arrowhead=crowodot]
    e -> E [arrowhead=crowtee]
    d -> D [arrowhead=teeodot]
    c -> C [arrowhead=teetee]
    b -> B [arrowhead=crow]
    a -> A [arrowhead=tee]
}

ellson/MOTHBALLED-graphviz#1266

Currently, only INTEGER and TEXT types are translated; we need to add support for more of Room data type definitions, including dates, blobs and etc.

Associated resources:

SQLite Data types

SQL Data Types:

Pull out DBML model classes into shared module that Room translator can depend on, in that way we can rely on a single definition of the DBML models across all modules of the codebase.

Potentially rename :core to something specific to Room, since we want to allow for other input formats (see for example: #19).

Clikt (pronounced “clicked”) is a multiplatform Kotlin library that makes writing command line interfaces simple and intuitive. It’s the “Command Line Interface for Kotlin”.

https://ajalt.github.io/clikt/

By using clikt, we can simplify the floorplan-cli module. On top of that, we get some general command-line standardisation and potential for more features.

Output files stored within directories and preserve structure

• For example, <input_dir>/com.mypackage.mydatabase/1.json should end up as <output_dir>/com.mypackage.mydatabase/1.dbml

Instead of writing out to DBML, could we render er files instead? Tools like erd can render image files via GraphViz (which, additionally, has java bindings).

So far there is no license file, could you please add it?

As an idea, would it be interesting to have an IntelliJ Plug-in, in which we can provide a way to run FloorPlan as an action in the IDE?

The Plug-in can detect file types that can be consumed by FloorPlan and provide a menu action / gutter action to generate the database schema in SVG/PNG and automatically render that in a new IDE tab.

Here is a great talk (slides) introducing the APIs and good sample projects to take inspiration from.

Room supports DEFAULT values, we should consider these and render them a part of DBML type notes.

Add DBML table notes using the creation SQL script exposed by Room's exported schema.

This should be toggled as an optionally enabled command line argument since SQL can be quite verbose and would in anycase 'duplicate' what the UI rendering for the DBML potentially is.

Try out the Provider API

• Example
• instead of String for the directory we can declare the field as DirectoryProperty and then to pass it, use something like project.layout.projectDirectory.dir("schemas"), so that the actual file will be resolved when the task runs

We are already capturing aliases and table notes in the Table definition for dbml-parser, but we don't include it in the output model.

In short: can the gradle plugin be told to skip the json-consumption part and process dbml input files?

I've been able to get the CLI to process DBML files as an input just fine

git clone [email protected]:julioz/FloorPlan.git   
git checkout tags/v0.3
./gradlew run --args="/home/daniel/Downloads/sample.dbml --format=dbml,svg,png,dot --output /home/daniel/workspaces/plaything/FloorPlan/oot --notation crowsfoot" 

But I can't get the gradle plugin to do this. My build.gradle is as follows

(using gradle wrapper; distributionUrl=https://services.gradle.org/distributions/gradle-6.6-bin.zip)

buildscript {
    repositories {
        google()
        jcenter()
    }

}
plugins {   
    id "base"
    id "com.juliozynger.floorplan" version "0.3"
}

floorPlan {
    schemaLocation = "$projectDir/schemas".toString()
    outputLocation = "$projectDir/schema-diagrams".toString()
    notation = "crowsfoot"
    outputFormat {     
        dbml {
            enabled = true
            creationSqlAsTableNote = false
            renderNullableFields = true
        }        
        svg {
            enabled = true
        }
        dot {
            enabled = true
        }                
    }
}

I can see in the logs it's saying

FloorPlan could not find any schema in specified location /home/daniel/workspaces/plaything/dbml-renderer/schemas

I think, because, the underlying FloorPlanConsumerSniffer only accepts json / db files.

This page here https://julioz.github.io/FloorPlan says this:

DBML parser
With a few manual tweaks, DBML can also be used directly as an input to the rendering mechanism, by skipping the pipeline's JSON-consumption step.

I can't see how the gradle plugin can be told to skip the json-consumption part, can it?

Thanks.

So far, dbml-parser doesn't properly handle indexes defined in DBML tables, as such:

Table bookings {
  id integer
  country varchar
  booking_date date
  created_at timestamp

  indexes {
      (id, country) [pk] // composite primary key
      created_at [note: 'Date']
      booking_date
      (country, booking_date) [unique]
      booking_date [type: hash]
      (`id*2`)
      (`id*3`,`getdate()`)
      (`id*3`,id)
  }
}

There are 3 types of index definitions:

Index with single field (with index name): CREATE INDEX on users (created_at)
Index with multiple fields (composite index): CREATE INDEX on users (created_at, country)
Index with an expression: CREATE INDEX ON films ( first_name + last_name )
(bonus) Composite index with expression: CREATE INDEX ON users ( country, (lower(name)) )

Running

floorplan-cli -v

or

floorplan-cli --version

should output the version that was used to compile the build

Currently, FloorPlan only allows for outputting one format at a time.
We want to enable multi format output, so that users can mix-and-match renderings as they seem fit.

The gradle plugin interface will then allow for multiple output enablement:

floorPlan {
  schemaLocation = ...
  outputLocation = ...
  outputFormat {
    dbml {
      enabled = true
    }
    svg {
      enabled = true
    }
  }
}

The CLI interface needs to also be rethought, to extend the --format argument to receive a List instead of a String. The curious bit is how we can make "nested arguments", given DBML can receive its own sub-parameters for rendering.

Hi, @julioz , @tomasz-moscicki-sonalake , I'd like to report a vulnerability issue in com.juliozynger.floorplan:floorplan-gradle-plugin:0.4.

Issue Description

com.juliozynger.floorplan:floorplan-gradle-plugin:0.4 directly or transitively depends on 26 C libraries (.so) cross many platforms(such as x86-64, x86, arm64, armhf). However, I noticed that one C library is vulnerable, containing the following CVEs:

libj2v8_linux_x86_64.so from C project openssl(version:1.0.2g) exposed 10 vulnerabilities:
CVE-2021-3712, CVE-2020-1968, CVE-2016-8610, CVE-2016-2182, CVE-2016-2181, CVE-2016-2179, CVE-2016-6302, CVE-2016-6303, CVE-2017-3738, CVE-2019-1552

Furthermore, the vulnerable methods in these vulnerable shared libraries can be actually invoked by Java code.
For instance, the following call chain starting from SSL_CTX_load_verify_locations() can reach the vulnerable method EC_GROUP_new_from_ecparameters() <EC_GROUP *EC_GROUP_new_from_ecparameters (const ECPARAMETERS *params) in crypto/ec/ec_asn1.c reported by CVE-2021-3712:

call chain -----
SSL_CTX_load_verify_locations() -> X509_STORE_load_locations() -> X509_STORE_add_lookup() -> STACK_OF() -> PEM_X509_INFO_read_bio() -> d2i_ECPrivateKey() -> EC_GROUP_new_from_ecpkparameters() -> EC_GROUP_new_from_ecparameters()

Suggested Vulnerability Patch Versions

openssl has fixed the vulnerabilities in versions >=1.1.1l

Java build tools cannot report vulnerable C libraries, which may induce potential security issues to many downstream Java projects.
Could you please upgrade the above shared libraries to their patch versions?

Thanks for your help~
Best regards,
Helen Parr

Besides Room, another popular library used in the Android ecosystem is SQLDelight. If we can generate schemas (or parse the .sq files) to use as input for generating DBML, we can increase the applicability of FloorPlan.

--args=~/foo/bar/1.json

but

--args=~/Users/me/foo/bar/1.json works.

Instead of printing the conversion output, passing an output file path argument to the program will write the DBML result to disk.