theosotr / cynthia Goto Github PK

View Code? Open in Web Editor NEW

14.0 14.0 1.0 2.03 MB

Data-Oriented Differential Testing of ORM Systems.

License: GNU General Public License v3.0

Python 2.34% Shell 1.52% Dockerfile 1.07% Scala 94.98% Makefile 0.09%

activerecord bugs differential-testing django orms peewee sequelize sqlalchemy testing

cynthia's Issues

Run queries on new backends

investigate if it is possible to run queries on new backends e.g.,

Oracle DB https://www.oracle.com/database/technologies/appdev/xe.html,
MS server https://www.microsoft.com/en-us/sql-server/sql-server-downloads
CockroachDB https://www.cockroachlabs.com/
MariaDB https://mariadb.org/ (Most ORMs use MySQL adapter for MariaDB)

SchemaTranslator: Quote table and column names

Generate string containing the '%' character

This may be useful to check whether ORM implement escaping properly.

Use constraint solvers for data generation

Peewee Translator: Fix issue with distnct

Implicitly add all expressions to the select list.

Systematically test the ActiveRecord Translator

Currently, the activerecord translator produces incorrect ORM queries when translating complex AQL queries, something that prevents us from testing activerecord.

We have to systematically test this translator through the test command on different backends, identify and fix the issues.

Implement a translator and model generator for PonyORM

Implement Translator for ActiveRecord ORM

Possible ORM candidates

https://github.com/hibernate/hibernate-orm (Java)
- Auto generate models
- Support Field Expressions
https://github.com/dresende/node-orm2 (Javascript)
- Auto generate models
- Support Field Expressions
https://github.com/go-gorm/gorm (Go)
- Auto generate models
- Support Field Expressions
https://github.com/diesel-rs/diesel (Rust)
- Auto generate models
- Support Field Expressions
https://github.com/doctrine/orm (PHP)
- Auto generate models
- Support Field Expressions
https://github.com/slick/slick (Scala)
- Auto generate models
- Support Field Expressions

Generate well-typed AQL expressions

Currently, we generate arbitrary expressions without checking the types of sub-expressions. The postgres SQL dialect is typed, so many of the generated AQL queries are doomed to fail (e.g., for example when mixing char fields with interger fields).

We should add a CLI option (e.g., --well-typed) to generate well-typed AQL expressions.

Run and Replay commands override existing results

When I run replay on a specific schema, and mismatch, it overrides existing mismatches.

For example, when I run:

cynthia replay --orms peewee,django -s Schema --mismatches 3

it produces results inside .cynthia/report/Schema/mismatches/1 rather than .cynthia/report/Schema/mismatches/3.

For run mode, cynthia should get the next available ID for assigning to the given query.

New AQL features

INSERT, UPDATE, and DELETE queries
distinct records
many-to-many relationships
one-to-one relationships
NULL values
Array values
Datetime values
More predicate operators. Some examples
- startswith
- endswith
- in
- is
- between
- ... more

+--------------------+
| Database           |
+--------------------+
| information_schema |
| Acclaimed          |
| Agism              |
| Airlifts           |
| Alexander          |
| Alkalis            |
| Amidst             |
| Annotates          |
| Apes               |
| Applaud            |
| Appraised          |
| Aqueduct           |
| Asoka              |
| Asperger           |
| Barreling          |
| Bawling            |
| Blaming            |
| Bloodbath          |
| Bouffant           |
| Bragged            |
| Brinks             |
| Bummer             |
| Bumpiest           |
| Burbling           |
| Cablecast          |
| Calamity           |
| Cane               |
| Carapace           |
| Carrier            |
| Catalysis          |
| Chirico            |
| Choppers           |
| Clinical           |
| Clothed            |
| Coachmen           |
| Cohesion           |
| Colors             |
| Condoes            |
| Contrives          |
| Corduroys          |
| Coverall           |
| Curtseys           |
| Cypriot            |
| Damage             |
| Darning            |
| Deeming            |
| Definable          |
| Deviance           |
| Diabetics          |
| Difficult          |
| Dilution           |
| Dishwater          |
| Dismaying          |
| Doubter            |
| Dreams             |
| Driver             |
| Drunkard           |
| Duelists           |
| Dusk               |
| Elena              |
| Enact              |
| Enlivens           |
| Ensnares           |
| Epicurus           |
| Erick              |
| Erupts             |
| Evens              |
| Exam               |
| Exchequer          |
| Expedient          |
| Expedites          |
| Expel              |
| Expensive          |
| Facials            |
| Fanatic            |
| Felicity           |
| Festive            |
| Fireworks          |
| Fisherman          |
| Fringe             |
| Frumpiest          |
| Fuchsias           |
| Funnily            |
| Furiously          |
| Gable              |
| Galilean           |
| Gates              |
| Ghettoes           |
| Ginseng            |
| Girded             |
| Glad               |
| Glitch             |
| Glottis            |
| Godot              |
| Goobers            |
| Gore               |
| Grazing            |
| Grille             |
| Grungier           |
| Gyp                |
| Hallmarks          |
| Handouts           |
| Harrowing          |
| Hoffman            |
| Homonym            |
| Humberto           |
| Ice                |
| Immensely          |
| Impeding           |
| Implants           |
| Insisted           |
| Isinglass          |
| Jauntier           |
| Jeweled            |
| Jilt               |
| Knifed             |
| Lanka              |
| Larcenous          |
| Lattice            |
| Laymen             |
| Legalese           |
| Lenard             |
| Liege              |
| Lindy              |
| Linnet             |
| Liquifies          |
| Lists              |
| Looping            |
| Lopes              |
| Loving             |
| Methought          |
| Midday             |
| Mixed              |
| Moniker            |
| Mortise            |
| Nebraskan          |
| Nicknamed          |
| Nocturne           |
| Nominees           |
| Novelle            |
| Now                |
| Nuanced            |
| Nurses             |
| Nuzzle             |
| Opponents          |
| Originate          |
| Outlast            |
| Overdraws          |
| Palmists           |
| Panacea            |
| Panniers           |
| Pat                |
| Pearliest          |
| Peritonea          |
| Pertness           |
| Perverted          |
| Pettiest           |
| Pillaging          |
| Pitchfork          |
| Pits               |
| Pneumatic          |
| Potted             |
| Prattles           |
| Prayed             |
| Precious           |
| Present            |
| Proteus            |
| Pulpy              |
| Putnam             |
| Rainbow            |
| Raised             |
| Reachable          |
| Reinforce          |
| Rejoining          |
| Rhodes             |
| Ricardo            |
| Riley              |
| Robotics           |
| Ronstadt           |
| Rushes             |
| Saner              |
| Savvy              |
| Scaffold           |
| Scalene            |
| Scrip              |
| Selves             |
| Sentiment          |
| Sepal              |
| Seventies          |
| Sexton             |
| Sharpest           |
| Showbiz            |
| Siestas            |
| Sightread          |
| Simulate           |
| Skewing            |
| Skitter            |
| Skittish           |
| Smokiness          |
| Solecism           |
| Sparklers          |
| Spelt              |
| Sporting           |
| Storybook          |
| Strainer           |
| Strays             |
| Succeed            |
| Suffuses           |
| Sundries           |
| Tabooing           |
| Target             |
| Titter             |
| Tom                |
| Twisted            |
| Unbinds            |
| Unhorses           |
| Vela               |
| Velour             |
| Verbals            |
| Visitor            |
| Wanamaker          |
| Wang               |
| Weekended          |
| Winnowing          |
| Woodcuts           |
| Zilch              |
| book               |
| books              |
| erp_progIITest     |
| listing            |
| mysql              |
| performance_schema |
| proderp            |
| sys                |
+--------------------+
234 rows in set (0.05 sec)

theosotr / cynthia Goto Github PK

cynthia's People

Contributors

Stargazers

Watchers

Forkers

cynthia's Issues

Recommend Projects

Recommend Topics

Recommend Org