Giter Club home page Giter Club logo

stormberry's Introduction

Hi, I'm Kilian 👋

I'm passionate about everything Dart & Flutter related 💙.

Checkout my work here on Github or:


🎯 Featured Project

I'm currently working on a new web framework for Dart - logo jaspr.

It is designed to be SSR-first and has a Flutter-style component system, so that is has next to no learning curve for developers coming from Flutter 💪.

It is still in the beta stage and I appreciate any help or feedback.


📯 Recommended Read

Useless Tips for Dart you probably never need.

This 3-part series gives a fun and interesting deep-dive into the abyss of unknown or weird Dart language features. It is a special kind of love letter to Dart.

There exist a lot of great articles highlighting the most useful tips and tricks for Flutter and Dart. They provide great value for both beginners and advanced developers.

This article is not one of them.

This is about the useless tips, the weird quirks, the boring facts and not needed tricks. They are either so absurd you won't ever need them, or so abstruse you don't wish to need them. But they deserve some love, too.


stormberry's People

Contributors

abitofevrything avatar brex900 avatar easazade avatar fmorschel avatar jaumard avatar kylehayes avatar nirmalariyathilake avatar pingear avatar schultek avatar timwhiting avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

stormberry's Issues

Dupicate of notification payloads in PostgreSQLConnection notifications

I've faced a strange behaviour using your stormberry package. Trying to access to the Database.connection() instance of PostgreSQLConnection makes it impossible to completely use it. For some reason, it is always closed, therefore there is no chance to get access to notifications and messages of connection instance, therefore I can't get any notifications due to closed connection. It is really strange untill the point that everything internaly inside database works perfectly. Every query or transaction runs without any problems, using database directly, without connection().

If you say that I should try to just use database.open(), unfortunately, it doesn't work for me. Because, as I said before, query and other methods runs flawlesy, therefore connection is already oppened and I can't open already opened connection, but directly trying to use connection() is not working, it is always closed for some reason.

I tried to overcome this issue by manually creating a PostgreSQLConnection instance with the same port, host and other fields for the connection. And here, I tried to open and create new instance of it when it's closed, but it lead to issue with too many connected connections, as because I couldn't open PostgreSQLConnection internaly, as it could be done with Database in stormberry, therefore, trying to continuosly reopen or reinit the connection instance in many files led to that problem. I had a lot of attempts to extend myself and solve this issue overcomming it, but I have no more idea how to fix it.

Why Database.connection() instance can be always closed? How I can fix that and directly use database connection in order to have access to the notifications without manuall creating PostgreSQLConnection over an over again that finally leads to another issue with a lot of opened and connectied connection. Untill then, I hope your understood the issue and my problem. If something is unclear, please let me know and I will try to provide more details and context. Thanks!

CREATE UNIQUE INDEX give PostgreSQLSeverity.error 42P07

Hello)
I have multiple tables that have columns with the same name that must have unique values.
image
image
When I try to migrate I get this error
image
Since the error says that the name field already exists, but it exists in another table, but within the frame of this transaction, I believe this is an unintended option that should not lead to an error

updateOne/updateMany not working for v0.12.1

Package Version

stormberry: 0.12.1

Error logs

ERROR - 2023-03-07 16:48:35.668546
Asynchronous error
PostgreSQLSeverity.error 42883: operator does not exist: bigint = text Hint: No operator matches the given name and argument types. You might need to add explicit type casts. 
package:postgres/src/connection.dart 514:18      _PostgreSQLExecutionContextMixin._query
package:postgres/src/connection.dart 475:7       _PostgreSQLExecutionContextMixin.query
package:stormberry/src/core/database.dart 91:34  Database.query

Error produced at

The statement with values.add(r.id), it doesn't actually giving the id in int type. I've print out the runTimeType it was String.

If I try to parse it via int.parse() or int.tryParse() it gives the radix exception i.e. the signed and unsigned format of integers.

Same issues for any kind of call I make here, any function involving id as int will cause trouble.

    @override
  Future<void> update(List<UserUpdateRequest> requests) async {
    if (requests.isEmpty) return;
    var values = QueryValues();
    await db.query(
      'UPDATE "users"\n'
      'SET "first_name" = COALESCE(UPDATED."first_name", "users"."first_name"), "followers" = COALESCE(UPDATED."followers", "users"."followers"), "following" = COALESCE(UPDATED."following", "users"."following"), "posts" = COALESCE(UPDATED."posts", "users"."posts"), "last_name" = COALESCE(UPDATED."last_name", "users"."last_name"), "username" = COALESCE(UPDATED."username", "users"."username"), "email" = COALESCE(UPDATED."email", "users"."email"), "password" = COALESCE(UPDATED."password", "users"."password"), "bio" = COALESCE(UPDATED."bio", "users"."bio"), "birthday" = COALESCE(UPDATED."birthday", "users"."birthday"), "image_url" = COALESCE(UPDATED."image_url", "users"."image_url"), "cover_url" = COALESCE(UPDATED."cover_url", "users"."cover_url")\n'
     --> 'FROM ( VALUES ${requests.map((r) => '( ${values.add(r.id)}:int8, ${values.add(r.firstName)}:text, ${values.add(r.followers)}:_int8, ${values.add(r.following)}:_int8, ${values.add(r.posts)}:_int8, ${values.add(r.lastName)}:text, ${values.add(r.username)}:text, ${values.add(r.email)}:text, ${values.add(r.password)}:text, ${values.add(r.bio)}:text, ${values.add(r.birthday)}:timestamp, ${values.add(r.imageURL)}:text, ${values.add(r.coverURL)}:text )').join(', ')} )\n'
      'AS UPDATED("id", "first_name", "followers", "following", "posts", "last_name", "username", "email", "password", "bio", "birthday", "image_url", "cover_url")\n'
      'WHERE "users"."id" = UPDATED."id"',
      values.values,
    );
  }

Improve migration tool

  1. Accept database arguments.
    • currently all database parameters must be provided through env vars, which is not very friendly.
  2. Manual mode
    • output migrations into sql files to allow users to manually and partially update their database

Add custom SQL field for Views

My use case here is that I have a huge table with billions of entries. To improve a bit performance I want a view with let say the data since two years only. By allowing a custom field where I can add a custom SQL I would be able to do that.
Hope that's clear enough ^^ or let me know

Error: `Unhandled Exception: PostgreSQLSeverity.error 42P01: relation "use_cases_id_seq" does not exist`

Running my Flutter project and trying to insert a new row into my use_cases table throws this error:

Unhandled Exception: PostgreSQLSeverity.error 42P01: relation "use_cases_id_seq" does not exist

But when using pgAdmin4 I can easily do the same select it's trying to do (SELECT nextval('use_cases_id_seq') as \"id\") and it runs perfectly. I'm trying to learn how to use the package. The connection is using the postgres user.

@Model()
abstract class UseCase {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  String get identifier;
  String get name;
  String get briefDescription;
  String get description;
  String get preCondition;
}

Bad state: Future already completed

Hi Kilian Schulte!
First I would like to thank you for this package. It seems to me better package among its concurrents especially when we need relationships.

here some suggestion from me. may be they are already implemented but I could not find them.

  • default value in models accessors (to fill updatedAt, createdAt automatically and many other cases)
  • using postgresql data types in models (now string converts text which is unwanted in many cases)
  • Queries & Actions classes with table name in it (e. db.alerts.query(query, params) I would like to have 'alerts' in query without passing it)

Here the issue which makes me create an issue I noticed the strange behaviour.

@Model(
  indexes: [
    TableIndex(name: 'user_id_alerts_index', columns: ['user_id'])
  ],
  views: [#Complete, #Reduced],
)
abstract class Alert {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  @HiddenIn(#Reduced)
  User get user;

  String get message;
  String? get path;
  bool get isRead;
  bool get isImportant;
  int get statusIndex;
  DateTime get createdAt;
  DateTime get updatedAt;
}
final db = Database(
  host: '127.0.0.1',
  port: 5432,
  database: 'xyz',
  user: 'xyz',
  password: 'xyz',
  useSSL: false,
);

void main() async {
  try {
    seeds();
  } catch (e) {
    print(e);
  }
}

AlertInsertRequest _getAlertInsertRequestInstance({
  required int userId,
  required String message,
  required int statusIndex,
  required bool isImportant,
  String? path,
}) {
  return AlertInsertRequest(
    userId: userId,
    message: message,
    statusIndex: statusIndex,
    isImportant: isImportant,
    path: path,
    isRead: false,
    updatedAt: DateTime.now().toUtc(),
    createdAt: DateTime.now().toUtc(),
  );
}

void seeds() async {
  firstInsert();
  secondInsert();
}

void firstInsert() async {
  var notId = await db.alerts.insertOne(
    _getAlertInsertRequestInstance(
      userId: 1,
      message: 'Ride #3 rejected.',
      statusIndex: NotificationStatus.warning.v,
      isImportant: true,
    ),
  );
  print('1: $notId');
}

void secondInsert() async {
  var notId = await db.alerts.insertOne(
    _getAlertInsertRequestInstance(
      userId: 2,
      message: 'Passenger #1 rejected from ride #3.',
      statusIndex: NotificationStatus.warning.v,
      isImportant: false,
    ),
  );
  print('2: $notId');
}
Output Details

Bad state: Future already completed
#0 _AsyncCompleter.complete (dart:async/future_impl.dart:41:31)
#1 Database.cancelTransaction (package:stormberry/src/core/database.dart:139:27)
#2 Database.runTransaction (package:stormberry/src/core/database.dart:171:7)

#3 secondInsert (file:///Users/vfiruz/Desktop/rohtj/backend/rohtj_server_grpc/seed/test.dart:58:15)

if I comment out secondInsert then it works.
If I make the returning type Future for firstInsert, secondInsert functions and use await before them then it works
if you could to check it out or explain what was the reason.

The reason not to make future and await sequently is that I want to reduce server response.
run someInserts without waiting and return server response.
and wondering to use some instance of Database class, is it possible?

Thanks!

Problem with migration [BUG?]

Thanks, the dart community has been waiting for a database tool for a long time. But I got a problem.

Enviroment:
MacOS
Postgres over docker compose.
Proof of ip:
image

run_migration.sh

DB_HOST_ADDRESS=172.19.0.2
DB_PORT=5432
DB_NAME=todo_list_db
DB_USERNAME=postgres
DB_PASSWORD=password
echo $DB_HOST_ADDRESS
echo $DB_PORT
echo $DB_NAME
echo $DB_USERNAME
echo $DB_PASSWORD
dart run stormberry migrate -o=./migrations/  --db=$DB_NAME

script output

yona@MacBook-Air-Daniil todolistdemobackend % ./scripts/build_migrations.sh 
172.19.0.2
5432
todo_list_db
postgres
password
Database: connecting to todo_list_db at 127.0.0.1...
Error on connection: SocketException: Connection refused (OS Error: Connection refused, errno = 61), address = 127.0.0.1, port = 51920
Database: retrying connecting...
SocketException: Connection refused (OS Error: Connection refused, errno = 61), address = 127.0.0.1, port = 51921
#0      _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:682:35)
#1      _NativeSocket.connect (dart:io-patch/socket_patch.dart:948:12)
#2      _RawSocket.connect (dart:io-patch/socket_patch.dart:1815:26)
#3      RawSocket.connect (dart:io-patch/socket_patch.dart:21:23)
#4      Socket._connect (dart:io-patch/socket_patch.dart:2038:22)
#5      Socket.connect (dart:io/socket.dart:743:21)
#6      PostgreSQLConnection.open (package:postgres/src/connection.dart:188:32)
#7      Database._tryOpen (package:stormberry/src/core/database.dart:80:34)
<asynchronous suspension>
#8      MigrateCommand.run (package:stormberry/src/cli/runner.dart:115:5)
<asynchronous suspension>
#9      CommandRunner.runCommand (package:args/command_runner.dart:209:13)
<asynchronous suspension>
#10     main (file:///Users/yona/.pub-cache/hosted/pub.dartlang.org/stormberry-0.9.2/bin/stormberry.dart:12:5)
<asynchronous suspension>

yona@MacBook-Air-Daniil todolistdemobackend % 

It looks like the migration tool can't see the environment variables.

Also i have feature request. Can we get tool for calling migration from code. I think it will useful.

HiddenIn & ViewedIn syntax

Hey! First, I would like to thank you for your package, so far I really like it.
I currently struggle with @HiddenIn and @ViewedIn syntax. Let's say I have following entities:

  • Project
  • Task
  • Subtask

Example:

@Model(views: [...])
abstract class Project {
  List<Task> get tasks;
}

Project has many tasks:

@Model(views: [...])
abstract class Task {
  Project get project;
  List<Subtask> get subtasks;
}

and access to subtasks (for example from Task entity I need to show how many subtasks are in there.

@Model(views: [..])
abstract class Subtask {
  Task get task;
}

Problem:

I don't understand the @HiddenIn & @ViewedIn syntax in order not to get cyclic dependencies. Can you please provide me example how to add the Views in order to get List<Task> from Project and List<Subtask> from Task?
Thanks!

Easy upsert possible?

Similar to #37, but wanting this sort of behavior, is it possible to currently do an upsert without having to check for presence first?

Thank you !

PostgreSQLException (PostgreSQLSeverity.error : Query cancelled due to the database connection closing.)

Hi, thank you very much for your wonderful plugin. I'm facing a very strange problem.
After Postgres returns an error about the uniqueness of names, then the error about the lack of connection to the database is constantly returned.

Снимок экрана 2023-03-24 в 17 24 17

[log] PostgreSQLException (PostgreSQLSeverity.error 23505: duplicate key value violates unique constraint "__unique" Detail: Key (email, username)=(1, 1) already exists. Table: user_models Constraint __unique )
[log] #0 _PostgreSQLExecutionContextMixin._query
connection.dart:514
#1 _PostgreSQLExecutionContextMixin.query
connection.dart:475
#2 Database.query
database.dart:91

#3 _UserModelRepository.insert
user_model.schema.dart:41

#4 Database.runTransaction
database.dart:163

#5 AuthMethodsService.signUp
auth_methods_service.dart:56

#6 new Stream.fromFuture. (dart:async/stream.dart:247:17)

[log] Error SignUp
[log] PostgreSQLException (PostgreSQLSeverity.error : Query cancelled due to the database connection closing. )
[log] Error SignUp
[log] PostgreSQLException (PostgreSQLSeverity.error : Query cancelled due to the database connection closing. )
[log] Error SignUp
[log] PostgreSQLException (PostgreSQLSeverity.error : Query cancelled due to the database connection closing. )
[log] Error SignUp

Upgrade upper limit to Analyzer

  • Dart SDK Version Dart SDK version: 2.18.6 (stable) (Tue Dec 13 21:15:14 2022 +0000) on "windows_x64"
  • What package(s) from this repo you are using, and the version
dependencies:
  flutter:
    sdk: flutter

  cupertino_icons: ^1.0.2
  stormberry: ^0.9.2

dev_dependencies:
  flutter_test:
    sdk: flutter
  
  build_runner: ^2.3.3
  dart_code_metrics: ^5.4.0
  flutter_lints: ^2.0.0

Output to pub get:

Running "flutter pub get" in my_project...                   
Because no versions of dart_code_metrics match >5.4.0 <6.0.0 and dart_code_metrics 5.4.0 depends on analyzer >=5.1.0 <5.4.0, dart_code_metrics ^5.4.0 requires analyzer >=5.1.0 <5.4.0.

unable to insert data

Hi,

I am using Dart_Frog, Stormberry, Postgres.

CREATE TABLE Product_Entity (
uid uuid DEFAULT uuid_generate_v4(),
title text NOT NULL, 
description text NOT NULL, 
price numeric NOT NULL, 
category text NOT NULL,
image text,
rating text,
PRIMARY KEY (uid)
);

CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

INSERT INTO Product_Entity (title, description, price, category) VALUES ('This is Title 1', 'This is description 1', 2.5, 'sports');
SELECT * from Product_Entity;

Above queries works perfectly. I am using postico2.

//////// NOW ////////////////////

Model class using Stormberry.

@Model(tableName: 'Product_Entity')
abstract class ProductEntity {
  @PrimaryKey()
  String? get uid;
  String get title;
  String get description;
  double get price;
  String get category;
}

After .schema.g.dart generated:

class ProductEntityInsertRequest {
  ProductEntityInsertRequest(
      {this.uid, required this.title, required this.description, required this.price, required this.category});
  String? uid;
  String title;
  String description;
  double price;
  String category;
}

///////// insert query looks like

@override
  Future<void> insert(Database db, List<ProductEntityInsertRequest> requests) async {
    if (requests.isEmpty) return;

    await db.query(
      'INSERT INTO "Product_Entity" ( "uid", "title", "description", "price", "category" )\n'
      'VALUES ${requests.map((r) => '( ${registry.encode(r.uid)}, ${registry.encode(r.title)}, ${registry.encode(r.description)}, ${registry.encode(r.price)}, ${registry.encode(r.category)} )').join(', ')}\n'
      'ON CONFLICT ( "uid" ) DO UPDATE SET "title" = EXCLUDED."title", "description" = EXCLUDED."description", "price" = EXCLUDED."price", "category" = EXCLUDED."category"',
    );
  }
  

Problem:

When trying to insert data using Dart_Frog as

 final request = ProductEntityInsertRequest(
      title: product.title,
      description: product.description,
      price: product.price,
      category: describeEnum(product.category),
    );
    await database.productEntities.insertOne(request);

It's throw this error:


INSERT INTO "Product_Entity" ( "uid", "title", "description", "price", "category" )
VALUES ( null, 'Product title 1', 'This is Product Description 1', 2.5, 'computers' )
ON CONFLICT ( "uid" ) DO UPDATE SET "title" = EXCLUDED."title", "description" = EXCLUDED."description", "price" = EXCLUDED."price", "category" = EXCLUDED."category"
[ERROR] 2022-12-22 23:22:57.797018 0:00:00.013230 POST /products

the issue is InsertQuery trying to insert null value in uid (ie: primaryKey)...

  1. When trying to use UUID as primary key, it's fail.
  2. When trying UUID , generated insertOne does NOT return anything....I guess, It should return uuid.

IDK, how to discard primary key field for InsertQuery, while using uuid_generate_v4 ?

Insert/Update ManyToMany relations

At the moment it is not possible to update ManyToMany , ManyToOne or OneToMany relations, because the insert/update models are missing the required attributes (and query). That is required, because there is no direct access to the JoinTables (in case of ManyToMany).

Example

// game_model.dart:
abstract class Game {
  @PrimaryKey()
  @AutoIncrement()
  int get id;
  List<Genre> get genres;
  @BindTo(#sequel)
  Game? get prequel;
  @BindTo(#prequel)
  Game? get sequel;
}

// genre_model.dart:
abstract class Genre {
  @PrimaryKey()
  @AutoIncrement()
  int get id;
  String get name;
  String? get description;
  List<Game> get games;
}

generates:

// game_model.schema.dart:
// [...]
class GameInsertRequest {
  GameInsertRequest({
    this.prequelId,
  });

  int? prequelId;
}

class GameUpdateRequest {
  GameUpdateRequest({
    required this.id,
    this.prequelId,
  });

  int id;
  int? prequelId;
}

// genre_model.schema.dart:
// [...]
class GenreInsertRequest {
  GenreInsertRequest({
    required this.name,
    this.description,
  });

  String name;
  String? description;
}

class GenreUpdateRequest {
  GenreUpdateRequest({
    required this.id,
    this.name,
    this.description,
  });

  int id;
  String? name;
  String? description;
}

Type conversion fails for List<CustomModels> with nullable parameters

Hi,
I have an AddressModel class with fromJson expecting a type of Map<String, dynamic> as few parameters are nullable. I store List<AddressModel> in another model and I have the TypeConverter for AddressModel as follows:

class AddressConverter extends TypeConverter<Address> {
  const AddressConverter() : super('jsonb');

  @override
  Map<String, dynamic> encode(Address value) => value.toJson();

  @override
  Address decode(dynamic value) {
    if (value is Map<String, dynamic>) {
      return Address.fromJson(value);
    } else {
      log(value.toString(), name: "Address Parsing Error");
      throw Exception("Parsing error: Unable to parse");
    }
  }
}

Now, the issue is that when the type conversion happens, it expects a List<Object> but finds a List<dynamic>. The workaround is to manually change List<Object> to List<dynamic> in the PostgresBinaryEncoder().convert() function from the postgres package as follows:

case PgDataType.jsonbArray:
        {
          if (input is List<dynamic>)  // Here, List<Object> is changed to List<dynamic>
          {
            final objectsArray = input.map((v) => utf8.encode(json.encode(v)));
            return writeListBytes<List<int>>(objectsArray, 3802, (item) => item.length + 1,
                (writer, item) {
              writer.writeUint8(1);
              writer.write(item);
            });
          }
          throw FormatException(
              'Invalid type for parameter value. Expected: List<Object> Got: ${input.runtimeType}');
        }

The code runs fine after his change and the data entry is added to the database. I am not sure if any changes can be made in AddressConverter code to resolve this issue.

Flutter 3.7.9 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 62bd79521d (13 days ago) • 2023-03-30 10:59:36 -0700
Engine • revision ec975089ac
Tools • Dart 2.19.6 • DevTools 2.20.1

stormberry: ^0.13.0

Unable to generate model schema with @HiddenIn(<Symbol>)

For the given user model, I have annotated the password to be hidden in the Base view. Now, when I generate the code, the following error is thrown which results in the schema to be not being generated.


[WARNING] stormberry:schema on lib/src/models/user_admin/user_admin.dart:
Failed to build database schema:

Column field was annotated with "HiddenIn(Symbol("Base"))", which is not supported.
  - String password

[WARNING] stormberry:schema on lib/src/models/user_admin/user_admin.dart:
#0      FieldColumnElement.checkModifiers (package:stormberry/src/builder/elements/column/field_column_element.dart:55:7)
#1      new ColumnElement (package:stormberry/src/builder/elements/column/column_element.dart:59:5)
#2      new _FieldColumnElement&ColumnElement&NamedColumnElement (package:stormberry/src/builder/elements/column/field_column_element.dart)
#3      new FieldColumnElement (package:stormberry/src/builder/elements/column/field_column_element.dart:19:9)
#4      TableElement.prepareColumns (package:stormberry/src/builder/elements/table_element.dart:110:21)
#5      SchemaState.finalize (package:stormberry/src/builder/schema.dart:38:17)
#6      SchemaState.getForAsset (package:stormberry/src/builder/schema.dart:31:5)
#7      OutputBuilder.build (package:stormberry/src/builder/builders/output_builder.dart:25:25)
<asynchronous suspension>
#8      runBuilder.buildForInput (package:build/src/generate/run_builder.dart:52:7)
<asynchronous suspension>
#9      Future.wait.<anonymous closure> (dart:async/future.dart:522:21)
<asynchronous suspension>

This is the model class for the user

@Model(views: [#Base, #DB])
abstract class UserAdmin {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  String get userName;

  @HiddenIn(#Base)
  String get password;
}

Dart SDK version: 2.19.2 (stable) on "windows_x64"
stormberry: ^0.12.1

Missing documentation for QueryParams and custom query

Hey,

QueryParams
i can't get the query with QueryParams to work and your documentation dont metnions them.
Also i guess they offer some risk for SQL-Injections?

Here is an example on what i expected to work:

    // Check if user already exists
    final matchingUser = (await db.users.queryUsers(const QueryParams(
      where: 'email="[email protected]"',
    )))

Error: (Column »[email protected]« doesn't exist)

PostgreSQLSeverity.unknown 42703: Spalte »[email protected]« existiert nicht 
_PostgreSQLExecutionContextMixin._query                  package:postgres/src/connection.dart:513
_PostgreSQLExecutionContextMixin.query                    package:postgres/src/connection.dart:474
Database.query                                                              package:stormberry/…/core/database.dart:94

custom query
I think it would be awesome if you would add a minimalistic example to the documentation whre you explain about Queries (https://pub.dev/packages/stormberry#queries)

Thanks!

[Unexpected] Insert updates existing entry

If the insert statement runs into a conflict with the primary key it will update the conflicting entry with the new information.

The generated query looks like:

  @override
  Future<void> insert(Database db, List<UserInsertRequest> requests) async {
    if (requests.isEmpty) return;

    await db.query(
      'INSERT INTO "users" ( "id", "email", "password_hash", "register_date", "email_confirmed" )\n'
      'VALUES ${requests.map((r) => '( ${registry.encode(r.id)}, ${registry.encode(r.email)}, ${registry.encode(r.passwordHash)}, ${registry.encode(r.registerDate)}, ${registry.encode(r.emailConfirmed)} )').join(', ')}\n'
      'ON CONFLICT ( "id" ) DO UPDATE SET "email" = EXCLUDED."email", "password_hash" = EXCLUDED."password_hash", "register_date" = EXCLUDED."register_date", "email_confirmed" = EXCLUDED."email_confirmed"',
    );
  }

This is unexpected and potentially dangerous. When not using auto increment an missplaced id would override existing data. For example:
A table uses guid as their primary keys. A new user registers and generates the same uuid. Instead of an error the query just updates the existing user and therefore links all refrence tables to a new person while loosing the previous user at the same time.

To update a entry of the database the method update should be used exclusivly.

Separate model class in different files ?

Hello,

Is it planned to be able to separate model classes in different files ?
The goal is to be able to have 1 class/file to find it more easily when you have a lot of different classes.

Add support for serialization

The generated entity classes should support being used with a serialization package like dart_mappable or json_serializable.

Pagination support

I'm used to do pagination with limit and offset, having something build in would be huge addition as of today the only way is to write the SQL.

Default enum serialization

Default enum serialization would be good. It could maybe be configurable via the annotation, but I think a sensible default is to use the .name getter on enums and the EnumType.values.byName(stringToParse) static method to serialize it to/from strings. This way adding / removing enums is non-breaking. (Using the index would be breaking). However this means that changing the name is breaking. Which is why it might be good to make it configurable.

Self-join relations

I am having the issue of trying to create a 1:1 relation between the same table.

@Model()
abstract class Game {
  @PrimaryKey()
  @AutoIncrement()
  int get id;
  
  // [...]
  
  Game? get prequel;
  //Game? get sequel;  // GameA.sequel = GameB.prequel
}

This will generate a schema with two prequelIds (probably due to cyclic issues, because none are a List). If I also add the sequel it will result into three prequelIds and one sequelId.

Not sure, but perhaps this could be fixed by adding somekind of Join annotation (similar to NestJs Typeorm, Spring Boot JPA, ...) that allows specifying the join column/attribute:

@Model()
abstract class Game {
  @PrimaryKey()
  @AutoIncrement()
  int get id;
  
  // [...]
  
  @Join('sequel')
  Game? get prequel;
  @Join('prequel')
  Game? get sequel;
}

This annotation could also be expanded to allow joins on non-primary key columns.

Configurable Line Length for Generator

Basically the title of the issue, right now the builder outputs a line length of 120 which I approve of, but most repositories I work with unfortunately still with a default of 80.

Happy to contribute a pull request. Just creating this issue to track.

Problem with DateTime field

Error:

ConverterException: Cannot decode value of type DateTime to type DateTime, because a value of type String or num is expected.

I have added:

else if (d is DateTime) {
      return d;
    }

To _DateTimeConverter in model_registry.dart and then all good but not sure that's a good fix as I didn't deep dive into the code yet

Virtual views

That one is just for the future but would be nice if user have choice of having View as real database views or just virtual views where we just sub query the fields of the table. That would allow having Views also for database that doesn't support views natively and stormberry could then work on mobile with sqlite.

The method 'ThisViewQueryable' isn't defined for the type 'ThatViewQueryable'

Got two models, user and post each with 2 different views complete and reduced. very similar to the example
There are syntax errors in generated code

The method 'PostViewQueryable' isn't defined for the type 'UserViewQueryable'.
The name 'PostView' isn't a type, so it can't be used as a type argument

here are the models

@Model(views: [#CompletePost, #ReducedPost])
abstract class Post {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  String get title;

  String get content;

  @BindTo(#posts)
  @ViewedIn(#CompletePost, as: #ReducedUser)
  @HiddenIn(#ReducedPost)
  User get author;

  User get editor;
}

@Model(views: [#CompleteUser, #ReducedUser])
abstract class User {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  String get name;

  String get bio;

  @BindTo(#author)
  @ViewedIn(#CompleteUser, as: #ReducedPost)
  @HiddenIn(#ReducedUser)
  List<Post> get posts;
}

you can find generated schema file here. there are 3 lines with errors at the end I commented them

also you can find the complete code here you need to git checkout this commit though: db949bd

hint: errors will be different when I change the order of the models in the file

Default value for not null fields in migration?

Hey! Really love this package.

I have a question to the following scenario:

The user table needs to be extended with a new column. This new column must not be null.
In my example we add a bool field like that:

@Model()
abstract class User {
  // EXISTING SCHEMA
  @PrimaryKey()
  String get id;
  String get email;
  String get passwordHash;
  int get registerDate;

  // PENDING MIGRATION
  bool get emailConfirmed;
}

The build will succeed but the migration will fail when applying the changes:

Select a database to update: 
Database: connecting to  at 127.0.0.1...
Database: connected
Getting schema changes of 
=========================
++ COLUMN users.email_confirmed
=========================
Do you want to apply these changes? (yes/no): yes
Database schema changed, applying updates now:
---
ALTER TABLE "users"
ADD COLUMN "email_confirmed" bool NOT NULL
Transaction finished with error: PostgreSQLSeverity.unknown 23502: Spalte »email_confirmed« von Relation »users« enthält NULL-Werte Table: users Column: email_confirmed 
========================
---
ALL CHANGES REVERTED, EXITING

This is to be expected because stormberry would need to update every existing entry with a default value.
Therefore my question now, would it be possible to define default values for fields so that they:

  1. are optional in the constructor
  2. the migration could automaticly update the database if we add a new not null collumn to a table with existing entries.

Hope i got this right :)

Cheers

Connection pool support

Hey, I'm checking for a Postgres ORM and found your package, really like your approach.
The query capabilities are quite light for now, having something like users..where.id.equals(10)..where.name.like('%Jim') a bit like what angel3_orm is doing is quite nice imho.

I don't see any support for connection pool, is it something you will consider? Would you accept pull request?

[Feature] Mark field as autoincrement

Hi, i was unable to mark field as autoincrement, so, i decided to do it with custom query
We definitively need this if we generate scheme of our db, cuz many apps/projects/etc often use autoincrement primary key 'id' (or something like this)

[Issue] Escaping variables with '

Hi, there's mistake with strings which contains ' in their body, because of this, the SQL query is terminated

'{"Brain' s Base "}' - ' after Brain cause error

I think it MUST be fixed, cuz it really bad vulnerability

Build runner error when attempting to generate simple one-to-many relationship

A simple reproducible example is:

@Model()
abstract class Foo {
  @PrimaryKey()
  String get id;
  List<Bar> get bars;
}

@Model()
abstract class Bar {
  @PrimaryKey()
  String get id;
}

This results in the following build_runner error:

Failed to build database schema:

type 'ReferenceColumnBuilder' is not a subtype of type 'ForeignColumnBuilder' of 'value'

why is the isClosed true after successfully connected

I have a problem in the production for reason. There is an issue almost like this issue#57 one. When occur any issue like passing wrong column name or smthg like that and after that sometimes we get closed connection and next queries fall with error Due to closed connection. As I declare my db variable final so I can fix my issue by reloading server. Now I want to check connection state before get Database instance and decide to whether reinitiate connection or not.

void main() async {
  final db = Database(
        '''
  );

  await db.open();
  print('isClosed: ${db.connection().isClosed}');
}

Just for Notice
My postgres is running in a different Docker container and my server is running in a different one. When any error occurs on the postgres site, its docker restarts and my server loses connection. How I can check state and reinitiate connection? I am open for any solution

postgress_db:
    container_name: "rohtj_db"
    image: postgres:16
    restart: always
    volumes:
      - ./rohtj_db/data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: rohtjdb
      POSTGRES_USER: rohtjdbuser
      POSTGRES_PASSWORD: ------
      PGDATA: /var/lib/postgresql/data/pgdata
    ports:
      - 5434:5432
    expose:
      - 5434
    networks:
      - rohtj-backend

Cant Update One nor Update Many

Im having this error when I try to update:
"An exception occurred.
PostgreSQLException (PostgreSQLSeverity.error 42883: operator does not exist: integer = text Hint: No operator matches name and types of arguments. Explicit type conversion may need to be added.)"

This is my call:
image

This is my model:
image

I did the migration 4 times already, I tried to workaround this by making a get, delete and insert, but I cant specify the Id in the insert either. Am I doing something wrong?

ConverterException: Parameter likes is not a List

I've added just a List<int> likes into the Post model. Then added some dummy data in postgres SQL.
But it seems that stormberry isn't parsing the list correctly.

The rest of the models are working fine, only with List in them are throwing this exception.
I'm using 0.11.0 latest version

ConverterException: Parameter likes is not a List
package:stormberry/src/internals/text_encoder.dart 137:7   TypedMap.getList
package:stormberry/src/internals/text_encoder.dart 147:12  TypedMap.getListOpt
package:db/src/models/post/post.schema.dart 143:18         PostQueryable.decode
package:stormberry/src/internals/view_query.dart 48:46     ViewQuery.apply.<fn>
dart:_internal                                             ListIterable.toList
package:stormberry/src/internals/view_query.dart 48:83     ViewQuery.apply

Model class and its schema file code is below:

post.dart

@Model()
abstract class Post {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  int get uid;
  String get caption;
  bool? get hasImage;
  String? get imageUrl;
  bool? get hasVideo;
  String? get videoUrl;
  List<int> get likes;
  List<Comment> get comments;

  DateTime get createdAt;
}

post.schema.dart

part of 'post.dart';

extension PostRepositories on Database {
  PostRepository get posts => PostRepository._(this);
}

abstract class PostRepository
    implements
        ModelRepository,
        KeyedModelRepositoryInsert<PostInsertRequest>,
        ModelRepositoryUpdate<PostUpdateRequest>,
        ModelRepositoryDelete<int> {
  factory PostRepository._(Database db) = _PostRepository;

  Future<Post?> queryPost(int id);
  Future<List<Post>> queryPosts([QueryParams? params]);
}

class _PostRepository extends BaseRepository
    with
        KeyedRepositoryInsertMixin<PostInsertRequest>,
        RepositoryUpdateMixin<PostUpdateRequest>,
        RepositoryDeleteMixin<int>
    implements PostRepository {
  _PostRepository(super.db) : super(tableName: 'posts', keyName: 'id');

  @override
  Future<Post?> queryPost(int id) {
    return queryOne(id, PostQueryable());
  }

  @override
  Future<List<Post>> queryPosts([QueryParams? params]) {
    return queryMany(PostQueryable(), params);
  }

  @override
  Future<List<int>> insert(List<PostInsertRequest> requests) async {
    if (requests.isEmpty) return [];
    var values = QueryValues();
    var rows = await db.query(
      'INSERT INTO "posts" ( "uid", "caption", "has_image", "image_url", "has_video", "video_url", "likes", "created_at" )\n'
      'VALUES ${requests.map((r) => '( ${values.add(r.uid)}:int8, ${values.add(r.caption)}:text, ${values.add(r.hasImage)}:bool, ${values.add(r.imageUrl)}:text, ${values.add(r.hasVideo)}:bool, ${values.add(r.videoUrl)}:text, ${values.add(r.likes)}:_int8, ${values.add(r.createdAt)}:timestamp )').join(', ')}\n'
      'RETURNING "id"',
      values.values,
    );
    var result = rows.map<int>((r) => TextEncoder.i.decode(r.toColumnMap()['id'])).toList();

    return result;
  }

  @override
  Future<void> update(List<PostUpdateRequest> requests) async {
    if (requests.isEmpty) return;
    var values = QueryValues();
    await db.query(
      'UPDATE "posts"\n'
      'SET "uid" = COALESCE(UPDATED."uid", "posts"."uid"), "caption" = COALESCE(UPDATED."caption", "posts"."caption"), "has_image" = COALESCE(UPDATED."has_image", "posts"."has_image"), "image_url" = COALESCE(UPDATED."image_url", "posts"."image_url"), "has_video" = COALESCE(UPDATED."has_video", "posts"."has_video"), "video_url" = COALESCE(UPDATED."video_url", "posts"."video_url"), "likes" = COALESCE(UPDATED."likes", "posts"."likes"), "created_at" = COALESCE(UPDATED."created_at", "posts"."created_at")\n'
      'FROM ( VALUES ${requests.map((r) => '( ${values.add(r.id)}:int8, ${values.add(r.uid)}:int8, ${values.add(r.caption)}:text, ${values.add(r.hasImage)}:bool, ${values.add(r.imageUrl)}:text, ${values.add(r.hasVideo)}:bool, ${values.add(r.videoUrl)}:text, ${values.add(r.likes)}:_int8, ${values.add(r.createdAt)}:timestamp )').join(', ')} )\n'
      'AS UPDATED("id", "uid", "caption", "has_image", "image_url", "has_video", "video_url", "likes", "created_at")\n'
      'WHERE "posts"."id" = UPDATED."id"',
      values.values,
    );
  }
}

class PostInsertRequest {
  PostInsertRequest({
    required this.uid,
    required this.caption,
    this.hasImage,
    this.imageUrl,
    this.hasVideo,
    this.videoUrl,
    required this.likes,
    required this.createdAt,
  });

  int uid;
  String caption;
  bool? hasImage;
  String? imageUrl;
  bool? hasVideo;
  String? videoUrl;
  List<int> likes;
  DateTime createdAt;
}

class PostUpdateRequest {
  PostUpdateRequest({
    required this.id,
    this.uid,
    this.caption,
    this.hasImage,
    this.imageUrl,
    this.hasVideo,
    this.videoUrl,
    this.likes,
    this.createdAt,
  });

  int id;
  int? uid;
  String? caption;
  bool? hasImage;
  String? imageUrl;
  bool? hasVideo;
  String? videoUrl;
  List<int>? likes;
  DateTime? createdAt;
}

class PostQueryable extends KeyedViewQueryable<Post, int> {
  @override
  String get keyName => 'id';

  @override
  String encodeKey(int key) => TextEncoder.i.encode(key);

  @override
  String get query => 'SELECT "posts".*, "comments"."data" as "comments"'
      'FROM "posts"'
      'LEFT JOIN ('
      '  SELECT "comments"."post_id",'
      '    to_jsonb(array_agg("comments".*)) as data'
      '  FROM (${CommentQueryable().query}) "comments"'
      '  GROUP BY "comments"."post_id"'
      ') "comments"'
      'ON "posts"."id" = "comments"."post_id"';

  @override
  String get tableAlias => 'posts';

  @override
  Post decode(TypedMap map) => PostView(
      id: map.get('id'),
      uid: map.get('uid'),
      caption: map.get('caption'),
      hasImage: map.getOpt('has_image'),
      imageUrl: map.getOpt('image_url'),
      hasVideo: map.getOpt('has_video'),
      videoUrl: map.getOpt('video_url'),
      likes: map.getListOpt('likes') ?? const [],
      comments: map.getListOpt('comments', CommentQueryable().decoder) ?? const [],
      createdAt: map.get('created_at'));
}

class PostView with Post {
  PostView({
    required this.id,
    required this.uid,
    required this.caption,
    this.hasImage,
    this.imageUrl,
    this.hasVideo,
    this.videoUrl,
    required this.likes,
    required this.comments,
    required this.createdAt,
  });

  @override
  final int id;
  @override
  final int uid;
  @override
  final String caption;
  @override
  final bool? hasImage;
  @override
  final String? imageUrl;
  @override
  final bool? hasVideo;
  @override
  final String? videoUrl;
  @override
  final List<int> likes;
  @override
  final List<Comment> comments;
  @override
  final DateTime createdAt;
}

FormatException: Invalid type code in substitution variable '@2:bool'

stormberry: 0.12.1

The error because of dart type bool and postgres SQL type boolean.
As I'm using bool in my dart classes, so it's failing to bind that class or convert it into a SQL based query.

The moment I modify the model.schema.g file and replace all bool with boolean it start working.

Model Class

@Model()
abstract class Post {
  @PrimaryKey()
  @AutoIncrement()
  int get id;

  int get uid;
  String get caption;
  bool? get hasImage;
  String? get imageUrl;
  bool? get hasVideo;
  String? get videoUrl;
  List<int> get likes;
  List<int> get comments;

  DateTime get createdAt;
}

Schema Class

This is where I make change to fix this, replace bool with boolean and it will work.

  @override
  Future<List<int>> insert(List<PostInsertRequest> requests) async {
    if (requests.isEmpty) return [];
    var values = QueryValues();
    var rows = await db.query(
      'INSERT INTO "posts" ( "uid", "caption", "has_image", "image_url", "has_video", "video_url", "likes", "comments", "created_at" )\n'
      'VALUES ${requests.map((r) => '( ${values.add(r.uid)}:int8, ${values.add(r.caption)}:text, ${values.add(r.hasImage)}:bool, ${values.add(r.imageUrl)}:text, ${values.add(r.hasVideo)}:bool, ${values.add(r.videoUrl)}:text, ${values.add(r.likes)}:_int8, ${values.add(r.comments)}:_int8, ${values.add(r.createdAt)}:timestamp )').join(', ')}\n'
      'RETURNING "id"',
      values.values,
    );
    var result = rows
        .map<int>((r) => TextEncoder.i.decode(r.toColumnMap()['id']))
        .toList();

    return result;
  }

‼️ Error

ERROR - 2023-03-02 10:06:38.503874
POST /api/v1/posts/
Error thrown by handler.
FormatException: Invalid type code in substitution variable '@2:bool'
package:postgres/src/query.dart 382:9            new PostgreSQLFormatIdentifier
package:postgres/src/substituter.dart 140:28     PostgreSQLFormat.substitute.<fn>
dart:_internal                                   ListIterable.join
package:postgres/src/substituter.dart 157:8      PostgreSQLFormat.substitute
package:postgres/src/query.dart 75:40            Query.sendExtended
package:postgres/src/connection_fsm.dart 375:9   _PostgreSQLConnectionStateReadyInTransaction.processQuery
package:postgres/src/connection_fsm.dart 361:14  _PostgreSQLConnectionStateReadyInTransaction.awake
package:postgres/src/connection.dart 620:67      _PostgreSQLExecutionContextMixin._enqueue
package:postgres/src/connection.dart 521:15      _PostgreSQLExecutionContextMixin._query
package:postgres/src/connection.dart 475:7       _PostgreSQLExecutionContextMixin.query
package:stormberry/src/core/database.dart 91:34  Database.query

Update for fields subset

In my API I want to do a PATCH request. So I need to be able to do an update just for few fields.
For now the generate update method allow only the update all at once.

Using other Postgre datatypes

Is there a way of using other postgre data types than the defaults that comes from using (String, bool, int) dart data types?
For exemple I want to use Date instead of timestamp. Varchar(255) instead text and so on.
I tried to use CustomDataTypes but wihout success.

class DateConverter extends TypeConverter {
const DateConverter() : super('date');
...
}

@UseConverter(DateConverter())
DateTime get dtOpen

[Issue] Can't generate schemes from many files

Hi, error described in header
When i was tried to generate scheme from two files in build.yaml - it's create only one table (for item1)
So, i tried to disable item1 and build another time, but scheme patcher just dropped 'item1' table and created new table for item2

At the current moment, I decided to remove any 'table dropping' code from source code (just as temporary solution) and generate new table for each item in builder.yaml - it works

Seems like patcher can't recognize multiple schemes, i think it's error and it should be fixed

image

image

Feature: Generate client serialization classes

Something that serverpod.dev does that is pretty nifty is to generate model serialization code, and put the classes in a separate dart client package. It makes it really easy for a developer to pull the client package into a flutter app (or any client app). Until you use it, it's hard to appreciate what a game changer it is. It makes development much faster.

One thought might be to use something like Freezed to define the "model" class, with the proper annotations. Freezed gives you serialization, and a bunch of other nice features.

There are going to be properties of a freezed model that you don't want sent to the database. I think you could handle those using some kind of @ignore annotation (or re-use JsonKey?).

Food for thought..

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.