strapi / migration-scripts Goto Github PK
View Code? Open in Web Editor NEWCollection of Strapi Migration scripts
Collection of Strapi Migration scripts
Hi. We are trying to migrate from strapi v3.x to v4.x using the migration scripts and codemods.
And when we attempted it, we get the error as shown in the screenshot:
UnhandledPromiseRejectionWarning: TypeError: Cannot read property ‘layouts’ of null
at /opt/ramco-cements-website-v4/migration-scripts/v3-sql-v4-sql/migrate/migrateCoreStore.js:51:17
at Array.map ()
at migrateItems (/opt/ramco-cements-website-v4/migration-scripts/v3-sql-v4-sql/migrate/helpers/migrateFields.js:24:16)
at Object.migrateTables (/opt/ramco-cements-website-v4/migration-scripts/v3-sql-v4-sql/migrate/migrateCoreStore.js:45:27)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async migrate (/opt/ramco-cements-website-v4/migration-scripts/v3-sql-v4-sql/migrate/index.js:63:5)
at async f (/opt/ramco-cements-website-v4/migration-scripts/v3-sql-v4-sql/index.js:6:3)
(Use node --trace-warnings ... to show where the warning was created)
(node:3437836) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see [Command-line API | Node.js v19.4.0 Documentation](https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode)). (rejection id: 1)
(node:3437836) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
We are unable to proceed with the migration because of this. Kindly help. We reached out to support, filed a ticket in the forum here: https://forum.strapi.io/t/data-migration-issue-from-v3-sql-to-v4-sql/24873 but weren't able to get a reply on the same.
i have this problem by migrate from v3 to v4
can somebody help me to fix this problem or does somebody have any idea ?
thanks in advance
Migrating 25786 items from users-permissions_user to up_users
users-permissions_user batch #1
(node:13348) UnhandledPromiseRejectionWarning: error: insert into "up_users" ("academy_department", "address_city", "address_country", "address_street", "address_zip", "billing", "blocked", "cities", "comments", "confirmation_token", "confirmed", "created_at", "created_by_id", "customer_id", "customer_number", "deletion_at", "description", "email", "email_confirmed", "id", "industry", "international_area_code", "language", "name", "organization_size", "partner_academy", "password", "phone", "privacy_consent_at", "provider", "reset_password_token", "slug", "stripe_customer_id", "subscriptions", "terms_consent_at", "token_version", "type", "updated_at", "updated_by_id", "username", "website_url", "welcomed_at")
Add additional logging options to have a verbose output of what is actually being done:
When migrating relations, when one side of the relationship is a component, the id column of the component field in the link table is incorrectly generated in some cases. This is causing the migration of some component relations to be skipped.
For my specific case, my component name is teacher_notes
and the relation column name in the link table is teacher_notes_id
. However, the migration scripts is trying to migrate to a column in the link table called teacher_note_id
(which doesn't exist) so is skipped.
Hard to reproduce as it is project-specific.
The relation column name in the link table for a component should be in the format {component_name}_id
.
If applicable, add screenshots to help explain your problem.
migration-scripts/v3-sql-v4-sql/migrate/helpers/relationHelpers.js
Lines 70 to 72 in 9ce0362
^ This is where the column name for a relation is being generated.
Within Strapi itself, the name of the relation column in a link table for a component is generated in the format: {relation_model_name}_id
. However, in the migration scripts, for all cases, the column name for a relation in the link table is being formatted using the pluralize
library, leading to column names for some component relations being incorrect.
After following the steps to migrate database from v3 to v4, the scripts fails with the following error:
error: delete from "up_users_role_links" - relation "up_users_role_links" does not exist
Database should be fully migrated to v4.
yarn start
entire outputyarn run v1.22.19
$ node index.js
Migrating Core Store TBA
Migrating 17/29 items from core_store to strapi_core_store_settings
core_store batch strapi/strapi#1
Migrating Admin
Migrating 3 items from strapi_role to admin_roles
DBV4 ITEMS
strapi_role batch strapi/strapi#1
Migrating 5 items from strapi_administrator to admin_users
DBV4 ITEMS
strapi_administrator batch strapi/strapi#1
Migrating 12 items from strapi_users_roles to admin_users_roles_links
DBV4 ITEMS
strapi_users_roles batch strapi/strapi#1
Migrating 85 items from strapi_permission to admin_permissions
strapi_permission batch strapi/strapi#1
strapi_permission batch strapi/strapi#2
Migrating Users
Migrating 2 items from users-permissions_role to up_roles
DBV4 ITEMS
users-permissions_role batch strapi/strapi#1
Migrating 14/210 items from users-permissions_permission to up_permissions
users-permissions_permission batch strapi/strapi#1
Migrating 1069 items from users-permissions_user to up_users
(node:238514) UnhandledPromiseRejectionWarning: error: delete from "up_users_role_links" - relation "up_users_role_links" does not exist
at Parser.parseErrorMessage (/mnt/480gb/casion-projects/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/mnt/480gb/casion-projects/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/mnt/480gb/casion-projects/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/mnt/480gb/casion-projects/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:400:28)
at addChunk (internal/streams/readable.js:293:12)
at readableAddChunk (internal/streams/readable.js:267:9)
at Socket.Readable.push (internal/streams/readable.js:206:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:238514) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:238514) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
The error seems to originate in https://github.com/strapi/migration-scripts/blob/main/v3-sql-v4-sql/migrate/migrateUsers.js on line 64, where it tries to delete everything from the table up_users_role_links
but this table doesn't exist.
This output I got when trying to migrate
Migrating Core Store TBA
Migrating 59/79 items from core_store to strapi_core_store_settings
core_store batch #1
core_store batch #2
Migrating Admin
Migrating 3 items from strapi_role to admin_roles
strapi_role batch #1
Migrating 1 items from strapi_administrator to admin_users
strapi_administrator batch #1
Migrating 1 items from strapi_users_roles to admin_users_roles_links
strapi_users_roles batch #1
Migrating 135 items from strapi_permission to admin_permissions
strapi_permission batch #1
strapi_permission batch #2
strapi_permission batch #3
Migrating Users
Migrating 3 items from users-permissions_role to up_roles
users-permissions_role batch #1
Migrating 198/486 items from users-permissions_permission to up_permissions
users-permissions_permission batch #1
users-permissions_permission batch #2
users-permissions_permission batch #3
users-permissions_permission batch #4
Migrating 416 items from users-permissions_user to up_users
users-permissions_user batch #1
users-permissions_user batch #2
users-permissions_user batch #3
users-permissions_user batch #4
users-permissions_user batch #5
users-permissions_user batch #6
users-permissions_user batch #7
users-permissions_user batch #8
users-permissions_user batch #9
Migration custom 01_my-custom-migration.js
Migrate webhooks
Migrating 0 items from strapi_webhooks to strapi_webhooks
Migrate locales (i18n)
SOURCE TABLE i18n_locales DOES NOT EXISTS
Migrating components
Migrating files
Migrating 1881 items from upload_file to files
upload_file batch #1
upload_file batch #2
upload_file batch #3
upload_file batch #4
upload_file batch #5
upload_file batch #6
upload_file batch #7
upload_file batch #8
upload_file batch #9
upload_file batch #10
upload_file batch #11
upload_file batch #12
upload_file batch #13
upload_file batch #14
upload_file batch #15
upload_file batch #16
upload_file batch #17
upload_file batch #18
upload_file batch #19
upload_file batch #20
upload_file batch #21
upload_file batch #22
upload_file batch #23
upload_file batch #24
upload_file batch #25
upload_file batch #26
upload_file batch #27
upload_file batch #28
upload_file batch #29
upload_file batch #30
upload_file batch #31
upload_file batch #32
upload_file batch #33
upload_file batch #34
upload_file batch #35
upload_file batch #36
upload_file batch #37
upload_file batch #38
Migrating 1751 items from upload_file_morph to files_related_morphs
upload_file_morph batch #1
upload_file_morph batch #2
upload_file_morph batch #3
upload_file_morph batch #4
upload_file_morph batch #5
upload_file_morph batch #6
upload_file_morph batch #7
upload_file_morph batch #8
upload_file_morph batch #9
upload_file_morph batch #10
upload_file_morph batch #11
upload_file_morph batch #12
upload_file_morph batch #13
upload_file_morph batch #14
upload_file_morph batch #15
upload_file_morph batch #16
upload_file_morph batch #17
upload_file_morph batch #18
upload_file_morph batch #19
upload_file_morph batch #20
upload_file_morph batch #21
upload_file_morph batch #22
upload_file_morph batch #23
upload_file_morph batch #24
upload_file_morph batch #25
upload_file_morph batch #26
upload_file_morph batch #27
upload_file_morph batch #28
upload_file_morph batch #29
upload_file_morph batch #30
upload_file_morph batch #31
upload_file_morph batch #32
upload_file_morph batch #33
upload_file_morph batch #34
upload_file_morph batch #35
upload_file_morph batch #36
Migrating Models
Migrating 90 items from customer to customer
customer batch #1
/Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/migrate/migrateModels.js:182
const [createdAt, updatedAt] = modelDef.options.timestamps;
^
TypeError: boolean false is not iterable (cannot read property Symbol(Symbol.iterator))
at /Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/migrate/migrateModels.js:182:38
at Array.map (<anonymous>)
at migrateItems (/Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/migrate/helpers/migrateFields.js:25:6)
at migrate (/Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/migrate/helpers/migrate.js:122:27)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async migrateModels (/Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/migrate/migrateModels.js:181:5)
at async migrate (/Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/migrate/index.js:70:3)
at async f (/Users/javiercastro/git/migration-scripts/v3-sql-v4-sql/index.js:6:3)
Node.js v18.0.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Configure .env and do yarn start
Just work.
v3-sql-v4-sql (pg 10 to pg13)
3.6.8 -> 4.1.9
All table names for content types in v3 use a plural name, but in v4 some of them use a singular name instead.
Error:
DESTINATION TABLE countries DOES NOT EXISTS
DESTINATION TABLE timezones DOES NOT EXISTS
In v4 the tables for these are country and timezone respectively.
Migrate a content type with a plural name such as "countries" from v3 to a singular name in v4 "country".
Ability to map previous table names to new ones in config?
I'm not sure what the ideal solution is right now, I'm going to try and fix it in my script and I'll update this issue if I've made any progress.
Database schema in environment variables does not work properly.
Use a schema other than public and run the v3-sql-v4-sql script
To use the right schema
If applicable, add screenshots to help explain your problem.
UnhandledPromiseRejectionWarning: error: delete from "admin_permissions_role_links" - relation "admin_permissions_role_links" does not exist
Many of these errors when trying to run the script. If I prepend the table names in the code with the schema it works. The environment variable schema is "strapi"
In some cases, many-to-many relations are not properly migrated and data is missing.
It looks like it has something to do with which attribute is the dominant
one, or with the alphabetical order of the referenced content-types, or with the name of the table that stores the relations.
npx create-strapi-app@v3 my-project --quickstart
to generate an empty Strapi v3 projectnpm i
npm run build && npm run develop
to start Strapi v4 and let it generate the empty tables, kill process afterwardsnpm run develop
to start Strapi v4The restaurant_categories_links table contains the migrated many-to-many-relations.
With the "How to reproduce" guide in mind this might look like an artificial issue. But we're currently migrating a v3 project which has been in production for 1.5 years to v4. That project contains some many-to-many relations of all sorts and changing how these relations are defined while ensuring data integrity is something I'm a little afraid of :)
By pure chance I found out that some data is missing. I hope there isn't more :(
If you try to reproduce this issue, pay attention to step 2:
When I create the many-to-many relation in the category as described in the Quick Start Guide, the many-to-many relation gets properly migrated. Only when I create it the other way round the migration does not happen.
I suspect it has something to do with either the alphabetical order of the content-type name, or with the dominant
property of the attribute.
Although the dominant
property would be weird, because the doc mentions that that one is used for NoSQL databases only.
When it's like this the migration will work:
// category.settings.json
{
...
"attributes": {
...
"restaurants": {
"collection": "restaurant",
"via": "categories",
"dominant": true
}
}
}
// restaurant.settings.json
{
...
"attributes": {
...
"categories": {
"via": "restaurants",
"collection": "category"
}
}
}
But when it's like this the migration will not work:
// category.settings.json
{
...
"attributes": {
...
"restaurants": {
"via": "categories",
"collection": "restaurant"
}
}
}
// restaurant.settings.json
{
...
"attributes": {
...
"categories": {
"collection": "category",
"via": "restaurants",
"dominant": true
}
}
}
The blog_post_categories_localizations_links
table of one of my content types is not being properly populated. Only the first column blog_post_category_id
column populated but not the inv_blog_post_category_id
column. However for some content types with similar complexity this works (i also have other content types that don't work either).
this is the v4 content type definition:
{
"kind": "collectionType",
"collectionName": "blog_post_categories",
"info": {
"singularName": "blog-post-category",
"pluralName": "blog-post-categories",
"displayName": "Blog Post Category",
"description": ""
},
"options": {
"draftAndPublish": false
},
"pluginOptions": {
"i18n": {
"localized": true
}
},
"attributes": {
"name": {
"pluginOptions": {
"i18n": {
"localized": true
}
},
"type": "string",
"unique": true,
"required": true
}
}
}
the v3 content type definition:
{
"kind": "collectionType",
"collectionName": "blog_post_categories",
"info": {
"name": "Blog Post Category",
"description": ""
},
"options": {
"increments": true,
"timestamps": true,
"draftAndPublish": false
},
"pluginOptions": {
"i18n": {
"localized": true
}
},
"attributes": {
"name": {
"pluginOptions": {
"i18n": {
"localized": true
}
},
"type": "string",
"unique": true,
"required": true
}
}
}
The v3 database shows the following records (interestingly the column is named blog-post_id
and not as expected related_blog_post_id
. maybe this is a no-issue, could be because of the -
vs. _
:
The v4 database:
i'd expect the v4 table to be properly populated.
might be related to #29
Running the migration script duplicates the value of edit in core store plugin_content_manager_configuration_content_types::
for all collection types that have a dynamic component.
This is what my collection looks like in version 3 Strapi
As you can see it duplicates the collection view config values after running the migration script
The value of the json in field edit is duplicated for collections with dynamic components
"edit": [ [ { "name": "Title", "size": 6 }, { "name": "slug", "size": 6 } ], [ { "name": "start", "size": 6 }, { "name": "end", "size": 6 } ], [ { "name": "CalendarDates", "size": 12 } ], [ { "name": "PreviewMode", "size": 4 }, { "name": "AttendancePoll", "size": 4 }, { "name": "HomepageEvent", "size": 4 } ], [ { "name": "logo", "size": 6 }, { "name": "Background", "size": 6 } ], [ { "name": "Summary", "size": 12 } ], [ { "name": "Content", "size": 12 } ], [ { "name": "Title", "size": 6 } ], [ { "name": "Content", "size": 12 } ], [ { "name": "PreviewMode", "size": 4 }, { "name": "AttendancePoll", "size": 4 }, { "name": "HomepageEvent", "size": 4 } ], [ { "name": "Summary", "size": 12 } ], [ { "name": "ContentDate", "size": 4 }, { "name": "Background", "size": 6 } ], [ { "name": "CalendarDates", "size": 12 } ] ],
v16.14.0
yarn 1.22.17
3.6.10 — Community Edition
4.3.9
Postgres 13
Postgres 13
macOS Monterey 12.5.1
yarn start
I got this error when I run the command yarn start
/Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/migrateFiles.js:36
if (item.uid.startsWith('application::')) {
^
TypeError: Cannot read properties of undefined (reading 'startsWith')
at /Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/migrateFiles.js:36:20
at Array.reduce (<anonymous>)
at /Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/migrateFiles.js:34:47
at Array.map (<anonymous>)
at migrateItems (/Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/helpers/migrateFields.js:24:16)
at migrate (/Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/helpers/migrate.js:129:27)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Object.migrateTables (/Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/migrateFiles.js:33:3)
at async migrate (/Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/migrate/index.js:63:5)
at async f (/Users/mykolabreslavskyi/www/perfsol/migration-scripts/v3-sql-v4-sql/index.js:6:3)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
I added console.log here and additional check to if statment
Running Mongo -> Sql
[Error: insert into `haunts` (`Name`, `Slug`, `created_at`, `id`, `updated_at`) values ('SeaWorld San Diego’s Howl-O-Scream', 'sea-world-san-diego-s-howl-o-scream', '2021-03-14 22:04:41.060', 1, '2021-03-14 22:06:44.220') - SQLITE_CONSTRAINT: NOT NULL constraint failed: haunts.CurrentlyOperating] {
errno: 19,
code: 'SQLITE_CONSTRAINT'
}
Schema for that:
{
"kind": "collectionType",
"collectionName": "haunts",
"info": {
"name": "Haunts",
"description": ""
},
"options": {
"increments": true,
"timestamps": true,
"draftAndPublish": false
},
"attributes": {
"Name": {
"type": "string",
"required": true
},
"Slug": {
"type": "uid",
"targetField": "Name",
"required": true
},
"AlternativeNames": {
"type": "component",
"repeatable": true,
"component": "common.alternative-names"
},
"CurrentlyOperating": {
"type": "boolean",
"default": true,
"required": true
},
"Geo": {
"type": "json"
},
"IsTempClosed": {
"type": "boolean",
"default": false,
"required": true
},
"Park": {
"model": "parks"
},
"NiceName": {
"type": "string"
},
"MainPhoto": {
"model": "file",
"via": "related",
"allowedTypes": [
"images"
],
"plugin": "upload",
"required": false
},
"CoverPhoto": {
"model": "file",
"via": "related",
"allowedTypes": [
"images"
],
"plugin": "upload",
"required": false
},
"Gallery": {
"collection": "file",
"via": "related",
"allowedTypes": [
"images"
],
"plugin": "upload",
"required": false
},
"Sections": {
"type": "dynamiczone",
"components": [
"text.section-header",
"text.text"
]
},
"Region": {
"model": "regions",
"via": "Haunts"
},
"OpeningDate": {
"type": "string"
},
"ClosedDate": {
"type": "string"
},
"HauntYears": {
"via": "Haunt",
"collection": "haunt-years"
},
"Attribute": {
"type": "component",
"repeatable": true,
"component": "attraction.attribute"
}
}
}
v16.17.0
v8.15.0
v3.5.0
v4.5.5
MySQL
MySQL
Windows 10 64-bit
.env
files)Please see here: https://github.com/strapi/migration-scripts/blob/main/v3-sql-v4-sql/migrate/helpers/relationHelpers.js#L49-L50
ManyToMany relationships in CollectionTypes with hyphens (e.g. Care-roles
) are not migrated properly (screenshots in the full description below).
The following lines need to have the value of modelF
and attributeF
wrapped in snakeCase()
, like so:
addRelation(
{
uid,
model: collectionName,
attribute: key,
type: 'manyToMany',
modelF: snakeCase(value.collection),
attributeF: snakeCase(value.attribute),
},
relations
);
Through testing on my side, wrapping just modelF
seems to work. However, no better / worse behavior was observed when adding it to attribtueF
, so ideally I'd need a second pair of eyes from the Strapi team to check for any consequences when adding it to the attributeF
field.
I have two CollectionTypes that interact with one another, articles
and care-roles
. They hold a many-to-many relationship, where many articles
can hold many care-roles
.
When I run the migration scripts, I seem to lose all relationships relating to the care-roles
collection. I would expect to see something like this in the migration script console output, when run:
Migrating X items from articles_care_roles__care_roles_articles to articles_care_roles_links
However, no message appears. This means the migration script has skipped over it for some reason. I spent a few hours debugging the migration script in VSCode line-by-line, and found the cause of it.
As shown in this screenshot, this will mean that the migrations script is looking for a table called articles_care-roles__care-roles_articles
(note the hyphens), which doesn't exist, and so is skipped over.
Whilst looking for potential fixes to this, I could see that the one-to-many addRelation()
function uses snakeCase()
. This sounded like something I needed for my many-to-many relationship. I added it like so and re-ran the script:
This seemed to fix the issue and all of my relationships migrated properly upon re-run! I ran this fix twice, once with just the modelF
value snakecased, and once with both modelF
and attributeF
snakecased.
There appears to be no difference in behavior; both fixed the issue properly, which makes me wonder if I'm potentially causing any other issues under-the-hood with this.
It would be much appreciated if I could have this reviewed by someone on the Strapi team, as I've found the solution and would like this to be reviewed and a fix merged into the repo, but I've been trying to contact the Strapi team for ~3 weeks to raise this issue, to no avail.
The Care-roles relationships should be migrated properly, outputting the following message in the console:
Migrating X items from articles_care_roles__care_roles_articles to articles_care_roles_links
Relevant screenshots included above, please see my messages in the Strapi discord for more screenshots:
Code samples AND pending solution posted above.
This is pretty-much solved and I just need this solution reviewed and merged into the main repo by a member of the Strapi team. The only reason I haven't made this a PR is because I'm unsure whether attributeF
requires the snake-case fix, so I'd like someone from the Strapi team to review the change to ensure no further issues happen with a result.
I'll be running this eventually on a real client's database, so having this fix officially merged in would be a real confidence-booster in running the script against their real database in a few weeks time.
When I try to migrate my Strapi v3 to v4, some many to many relations doesn't going right, after debug the code I found an issue.
So in script v3-sql-v4-sql
in the below part o code, the verification is incorrect.
Many to many relations doesn't has a column to relate, they has it's own table to make the relations. (I know in Strapi V4 all relations has it's own table, but if in migration the relation it's doesn't identified as many to many your data is gonna be wrong)
The part of code here
So, to fix it locally I do: if (value.column)
>>> if (!value.column) ...
1 - Strapi V3 with PG
2 - Has many to many relations in strapi V3
3 - Do the steps of migration described in https://github.com/strapi/migration-scripts
Your many to many relations gonna be empty.
Maybe can be related with this other issue
which in my case was the development database, and not production
I was trying to connect to. Took my a while before i realized what was happing 🥲
When old columns were not deleted (caused by: strapi/strapi#1114 most likely) the script will try to transfer data from these columns but since they don't exist on v4 the script crashes. We gracefully handle this for tables but not columns.
We should gracefully handle fields that exist in v3 database that don't in v4 and ideally warn the user that the field was missing and data was not transferred for it.
@martincapek If you need more info, if you are working on this, ping me on slack as I have a customer's project that I was troubleshooting and can help to show you how to reproduce this.
In the strapi v3 i have tables :
components_story_group_stories__stories
with columns id
| components_story_group_story_id
| story_id
components_puzzle_group_puzzles__puzzles
with columns id
| components_puzzle_group_puzzle_id
| puzzle_id
I migrated my strapi from v3 to v4 using codemods. And pointed strapi to new empty db and generated schema. After migrating the strapi following changes happen
components_story_group_stories_stories_links
with the columns id
| group_stories_id
| story_id
| story_order
components_puzzle_group_puzzles_puzzles_links
with columns id
| group_puzzles_id
| puzzle_id
| puzzle_order
**there are more like these tables
When i tried to migrate the data from v3 to v4 the column group_stories_id
and group_puzzles_id
are skipped
Showing this in the console -
WARNING - items of components_story_group_stories_stories_links was filtered ["group_story_id"]
WARNING - items of components_puzzle_group_puzzles_puzzles_links was filtered ["group_puzzle_id"]
In columns group_story_id
and group_puzzle_id
NULL
value is showing
The other issue with the same process is no data is migarted for some tables:
in v3 i have a table components_game_group_games__game_sets
with columns id
| components_game_group_game_id
| game-set_id
after migrating it to the v4 the table is renamed as components_game_group_games_game_sets_links
with columns name as id
| group_games_id
| game_set_id
| game_set_order
After running the migration script the table is still empty in v4 db and data for this table is not migrated from v3 to v4.
All the data should have been migrated from v3 to v4 PostgreSQL.
N/A, this is a current typo
Typo in code. Referencing 'attribute.moel' instead of 'attribute.model'
Reference attribute.model
If applicable, add screenshots to help explain your problem.
const isOneWay = attribute.model && !attribute.via && attribute.moel !== '*';
It starts the migration process, but after a while restoring records in a specific table it gets stuck and shows this error (regarding duplicate ids, but original data does not have duplicate ids of course since its the PK):
duplicate key value violates unique constraint "node_states_pkey"
at Parser.parseErrorMessage (/strapi-v3-v4-db-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/strapi-v3-v4-db-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/strapi-v3-v4-db-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/strapi-v3-v4-db-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (node:events:527:28)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
length: 204,
severity: 'ERROR',
code: '23505',
detail: 'Key (id)=(5440853) already exists.',
hint: undefined,
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: 'public',
table: 'node_states',
column: undefined,
dataType: undefined,
constraint: 'node_states_pkey',
file: 'nbtinsert.c',
line: '663',
routine: '_bt_check_unique'
}
error Command failed with exit code 1.
migration-scripts/v3-sql-v4-sql/migrate/helpers/migrate.js
Lines 153 to 161 in 9ce0362
On PostgreSQL (have not tried other backends), table references should use "
instead of '
.
yarn run v1.22.18
$ node index.js
Migrating Core Store
Migrating 18/32 items from core_store to strapi_core_store_settings
core_store batch #1
(node:64141) UnhandledPromiseRejectionWarning: error: SELECT SETVAL ('strapi_core_store_settings_id_seq', (SELECT MAX(id) + 1 FROM 'strapi_core_store_settings')) - syntax error at or near "'strapi_core_store_settings'"
at Parser.parseErrorMessage (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:315:20)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at Socket.Readable.push (internal/streams/readable.js:223:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:64141) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:64141) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
When running the scripts I get DESTINATION TABLE does not exist for all my content types; I cannot find any code that would actually create the DESTINATION tables when trying to troubleshoot. Is the concept of the migration scripts that the content types first get created in a manual manner in Strapi v4 and the migration does not migrate schemas but only data?
Destination tables being created for all content types based on source.
Taken 8hrs to get this far :-/
- Node.js version: v16.15.0
- Source Strapi version: 3
- Target Strapi version: 4
- Source Database: postgres
- Target Database: postgres
- Operating system: nixos
- Which script are you running: v3-sql-v4-sql
Migration failure
error: insert into "admin_permissions" ("action", "conditions", "created_at", "fields", "id", "properties", "subject", "updated_at") values ($1, $2, $3, $4, $5, $6, $7, $8), ($9, $10, $11, $12, $13, $14, $15, $16), ($17, $18, $19, $20, $21, $22, $23, $24), ($25, $26, $27, $28, $29, $30, $31, $32), ($33, $34, $35, $36, $37, $38, $39, $40), ($41, $42, $43, $44, $45, $46, $47, $48), ($49, $50, $51, $52, $53, $54, $55, $56), ($57, $58, $59, $60, $61, $62, $63, $64), ($65, $66, $67, $68, $69, $70, $71, $72), ($73, $74, $75, $76, $77, $78, $79, $80), ($81, $82, $83, $84, $85, $86, $87, $88), ($89, $90, $91, $92, $93, $94, $95, $96), ($97, $98, $99, $100, $101, $102, $103, $104), ($105, $106, $107, $108, $109, $110, $111, $112), ($113, $114, $115, $116, $117, $118, $119, $120), ($121, $122, $123, $124, $125, $126, $127, $128), ($129, $130, $131, $132, $133, $134, $135, $136), ......... - column "fields" of relation "admin_permissions" does not exist
Follow v3-sql-v4-sql guide and migrate.
A smooth transition from Strapi V3 to V4
I have an extra column on v3's strapi_permission
table. At the moment I'm not sure if it's being used by the lib or the project. What would be the best way to port it to v4.
I get this error when trying to migrate SQL data, v3 to v4:
column "fields" of relation "admin_permissions" does not exist
1.Follow the instruction from README file
2. See errors
Not to have this error because afaik there isn't any fields
column in v4's admin_permissions
table.
n/a
Longer stack:
yarn start
yarn run v1.22.18
$ node index.js
Migrating Core Store TBA
Migrating 100/197 items from core_store to strapi_core_store_settings
core_store batch #1
core_store batch #2
Migrating Admin
Migrating 9 items from strapi_role to admin_roles
strapi_role batch #1
Migrating 109 items from strapi_administrator to admin_users
strapi_administrator batch #1
strapi_administrator batch #2
strapi_administrator batch #3
Migrating 109 items from strapi_users_roles to admin_users_roles_links
strapi_users_roles batch #1
strapi_users_roles batch #2
strapi_users_roles batch #3
Migrating 871 items from strapi_permission to admin_permissions
strapi_permission batch #1
(node:66281) UnhandledPromiseRejectionWarning: error: insert into "admin_permissions" ("action", "conditions", "created_at", "fields", "id", "properties", "subject", "updated_at") values ($1, $2, $3, $4, $5, $6, $7, $8), ($393, $394, $395, $396, $397, $398, $399, $400) - column "fields" of relation "admin_permissions" does not exist
at Parser.parseErrorMessage (strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:400:28)
at addChunk (internal/streams/readable.js:293:12)
at readableAddChunk (internal/streams/readable.js:267:9)
at Socket.Readable.push (internal/streams/readable.js:206:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:66281) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:66281) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
n/a
The parameterized insert syntax for a json
field in PostgreSQL contains invalid syntax. Specifically, - invalid input syntax for type json
In context:
(node:67148) UnhandledPromiseRejectionWarning: error: insert into "public"."services" ("created_at", "created_by_id", "description", "high_points", "id", "name", "published_at", "updated_at", "updated_by_id") values ($1, $2, $3, $4, $5, $6, $7, $8, $9), ($10, $11, $12, $13, $14, $15, $16, $17, $18), ($19, $20, $21, $22, $23, $24, $25, $26, $27), ($28, $29, $30, $31, $32, $33, $34, $35, $36), ($37, $38, $39, $40, $41, $42, $43, $44, $45), ($46, $47, $48, $49, $50, $51, $52, $53, $54) - invalid input syntax for type json
at Parser.parseErrorMessage (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/Users/colearendt/working/strapi-migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:315:20)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at Socket.Readable.push (internal/streams/readable.js:223:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:67148) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:67148) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Migrate JSON data from v3 to v4
After running migration-scripts, when consulting the details of one entry, some fields appears twice.
Below for example for the Content-type : News-type, the field : 'type' appears twice
When the schema.json of News-type is :
"kind": "collectionType",
"collectionName": "news_types",
"info": {
"singularName": "news-type",
"pluralName": "news-types",
"displayName": "News-type",
"name": "news-type"
},
"options": {
"increments": true,
"timestamps": true,
"draftAndPublish": true
},
"pluginOptions": {
"i18n": {
"localized": true
}
},
"attributes": {
"type": {
"type": "string",
"required": true,
"pluginOptions": {
"i18n": {
"localized": true
}
}
}
}
}
I also checked the database and there is no double column 'type'. So it is just a problem in the view. See below, we have the Config view edit and the field 'type' appears twice for some reason.
Interestingly the 2 other Content-types which are linked to News-type have some problem in their view as well :
Latest-news-and-resource
{
"kind": "collectionType",
"collectionName": "news_and_resources",
"info": {
"singularName": "latest-news-and-resource",
"pluralName": "latest-news-and-resources",
"displayName": "Latest-news-and-resource",
"name": "latest-news-and-resource",
"description": ""
},
...
"attributes": {
"title": {
"type": "string",
"required": true,
"pluginOptions": {
"i18n": {
"localized": true
}
}
},
...
"news_type": {
"type": "relation",
"relation": "oneToOne",
"target": "api::news-type.news-type"
},
"primary_category": {
"type": "relation",
"relation": "manyToOne",
"target": "api::category.category",
"inversedBy": "primary_category_news_and_resources"
},
"other_categories": {
"type": "relation",
"relation": "manyToMany",
"target": "api::category.category",
"inversedBy": "other_categories_news_and_resources"
},
...
}
}
}
For Latest-news-and-resources, the fields : news-type, primary_category, other_categories . All appear twice as in the image below :
And it is the same for Category :
{
"kind": "collectionType",
"collectionName": "categories",
"info": {
"singularName": "category",
"pluralName": "categories",
"displayName": "Category",
"name": "category"
},
"options": {
"increments": true,
"timestamps": true,
"draftAndPublish": true
},
"pluginOptions": {
"i18n": {
"localized": true
}
},
"attributes": {
"name": {
"type": "string",
"required": true,
"pluginOptions": {
"i18n": {
"localized": true
}
}
},
"primary_category_news_and_resources": {
"type": "relation",
"relation": "oneToMany",
"target": "api::latest-news-and-resource.latest-news-and-resource",
"mappedBy": "primary_category"
},
"other_categories_news_and_resources": {
"type": "relation",
"relation": "manyToMany",
"target": "api::latest-news-and-resource.latest-news-and-resource",
"mappedBy": "other_categories"
}
}
}
It looks like this problem only appear for Content-types which have '-' hyphen in the name and is linked to other Content-types.
Once again the database and the model files are all fine, it is just in the view.
No fields should be displayed twice.
The duplicated field through the Config view edit of the Content Manager. But we would like to avoid any manual action because we aren't necessarily aware where this may appear and we aren't sure if it will appear again.
During model migration the script omits existing "created_at" and "updated_at" fields/values,
thus migrated models in target database have empty "created_at" and "updated_at" fields.
Migrated models in target database should have "created_at" and "updated_at" timestamps - matching the "created_at" and "updated_at" timestamps of models from source database.
n/a
see upcoming PR
A new issue was introduced by this commit:
471c641
When a strapi v3 column has a hyphen in its name, "inv_" is being prepended to the strapi v4 column name, causing the migration to fail.
See the comments added to the commit for the specific line causing the failure
v3-sql-v4-sql
If the U&P plugin is missing from the target project, the script will crash.
error: delete from "up_permissions_role_links" - relation "up_permissions_role_links" does not exist
Pretty straightforward, just try a migration, targeting a project without U&P installed.
I would expect a warning message explaining that the U&P plugin is probably not installed, and the users' permissions were not migrated. Apart from that everything should work nonetheless.
Or you can choose to stop there and tell the user to add U&P to his/her app but then it implies that U&P is not really a plugin as it's required.
Can't handle all locales from single type after migration from strapi 3
Fixed manually by run something like
insert into
blog_page_localizations_links (
blog_id,
inv_blog_id) values (2, 1) insert into
blog_page_localizations_links (
blog_id,
inv_blog_id) values (1, 2)
insert or update on table "x_components" violates foreign key constraint "x_entity_fk"
TODO: Add information about sql to sql script for Notum (Credit/link to their site/ect)
The conditions values in the admin_permissions tables are improperly formatted after running the migration script, this causes application crashes for admin users assigned to any role created pre-migration besides the super admin.
The dashboard does not crash and the server does not error.
I have the latest version with the Pull Request which fixed the relation issues #14 , but I still get an error that a column does not exist in a relation
error: insert into "components_slidetypen_template_twl_s_lights_links" ("components_slidetypen_template_twl_id", "light_id", "template_twl_id") values ($1, $2, $3), ($4, $5, $6), ($7, $8, $9), ($10, $11, $12) - column "components_slidetypen_template_twl_id" of relation "components_slidetypen_template_twl_s_lights_links" does not exist
The relation in question "lights" here is a 'hasMany' relation.
The source Model looks like this:
{
"collectionName": "components_slidetypen_template_twl_s",
"info": {
"name": "template_TWL_S",
"icon": "arrow-left",
"description": ""
},
"options": {},
"attributes": {
"title": {
"type": "string"
},
"subtitle": {
"type": "string"
},
"lights": {
"collection": "light"
},
"actorList": {
"type": "component",
"repeatable": true,
"component": "sensors.actor-group"
},
"sensorList": {
"type": "component",
"repeatable": true,
"component": "sensors.sensor-group"
}
}
}
The Target Model looks like this:
{
"collectionName": "components_slidetypen_template_twl_s",
"info": {
"name": "template_TWL_S",
"icon": "arrow-left",
"description": ""
},
"options": {},
"attributes": {
"title": {
"type": "string"
},
"subtitle": {
"type": "string"
},
"lights": {
"type": "relation",
"relation": "oneToMany",
"target": "api::light.light"
},
"actorList": {
"type": "component",
"repeatable": true,
"component": "sensors.actor-group"
},
"sensorList": {
"type": "component",
"repeatable": true,
"component": "sensors.sensor-group"
}
}
}
I also added the code snippets from #9. I thought it could have something to do with the Plural Naming, but it still does not work.
I am open for suggestions and questions.
We are pretty reliant on a v4 migration for the project to not launch it with an outdated version.
v16.17.0
v8.15.0
v3.5.0
v4.5.5
MySQL
MySQL
Windows 10 64-bit
.env
files)This appears to be the same issue as this bug from last year: #10
This issue was never properly solved and was closed due to inactivity. As all of the code that I'm using has been provided by the Strapi team, I'm hoping that this issue will provide a reproducible example, so the team can provide an explanation for the issue, and potentially a fix. For any repos / npm packages I use, I aim to provide commit IDs and version numbers.
This also looks like it might be related to #19
After using the Strapi example repo to migrate the demo strapi application from V3 to V4 via codemods (and following the livestream: https://www.youtube.com/watch?v=NSvdQKVvV9k), I've cloned the migration-scripts repo and followed the data migration livestream (https://www.youtube.com/watch?v=kdxivJjjVhY).
After filling in the env files and specifying my targets, I run the migration script, but then get the following error:
npm run start
> [email protected] start
> node index.js
Migrating Core Store
Migrating 59/88 items from core_store to strapi_core_store_settings
core_store batch #1
core_store batch #2
Migrating Admin
Migrating 3 items from strapi_role to admin_roles
strapi_role batch #1
Migrating 1 items from strapi_administrator to admin_users
strapi_administrator batch #1
Migrating 2 items from strapi_users_roles to admin_users_roles_links
strapi_users_roles batch #1
Migrating 100 items from strapi_permission to admin_permissions
strapi_permission batch #1
node:internal/process/promises:279
triggerUncaughtException(err, true /* fromPromise */);
^
[Error: insert into `admin_permissions` (`action`, `conditions`, `created_at`, `fields`, `id`, `properties`, `subject`, `updated_at`) select 'plugin::content-manager.explorer.create' as `action`, '[]' as `conditions`, 1595433593180 as `created_at`, '["name","restaurants"]' as `fields`, 1 as `id`, NULL as `properties`, 'api::category.category' as `subject`, 1595433593185 as `updated_at` union all select 'plugin::content-manager.explorer.create' as `action`, '[]' as `conditions`, 1595433593192 as `created_at`, '["author","review"]' as `fields`, 2 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593197 as `updated_at` union all select 'plugin::content-manager.explorer.create' as `action`, '[]' as `conditions`, 1595433593217 as `created_at`, '["content","note","author","likes","restaurant"]' as `fields`, 4 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593222 as `updated_at` union all select 'plugin::content-manager.explorer.read' as `action`, '[]' as `conditions`, 1595433593230 as `created_at`, '["name","restaurants"]' as `fields`, 5 as `id`, NULL as `properties`, 'api::category.category' as `subject`, 1595433593235 as `updated_at` union all select 'plugin::content-manager.explorer.read' as `action`, '[]' as `conditions`, 1595433593243 as `created_at`, '["author","review"]' as `fields`, 6 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593247 as `updated_at` union all select 'plugin::content-manager.explorer.read' as `action`, '[]' as `conditions`, 1595433593268 as `created_at`, '["content","note","author","likes","restaurant"]' as `fields`, 8 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593274 as `updated_at` union all select 'plugin::content-manager.explorer.update' as `action`, '[]' as `conditions`, 1595433593283 as `created_at`, '["name","restaurants"]' as `fields`, 9 as `id`, NULL as `properties`, 'api::category.category' as `subject`, 1595433593288 as `updated_at` union all select 'plugin::content-manager.explorer.update' as `action`, '[]' as `conditions`, 1595433593301 as `created_at`, '["author","review"]' as `fields`, 10 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593307 as `updated_at` union all select 'plugin::content-manager.explorer.update' as `action`, '[]' as `conditions`, 1595433593335 as `created_at`, '["content","note","author","likes","restaurant"]' as `fields`, 12 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593342 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593352 as `created_at`, NULL as `fields`, 13 as `id`, NULL as `properties`, 'api::category.category' as `subject`, 1595433593359 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593371 as `created_at`, NULL as `fields`, 14 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593377 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593386 as `created_at`, NULL as `fields`, 15 as `id`, NULL as `properties`, 'api::restaurant.restaurant' as `subject`, 1595433593391 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593403 as `created_at`, NULL as `fields`, 16 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593408 as `updated_at` union all select 'plugin::upload.read' as `action`, '[]' as `conditions`, 1595433593417 as `created_at`, NULL as `fields`, 17 as `id`, NULL as `properties`, NULL as `subject`, 1595433593422 as `updated_at` union all select 'plugin::upload.assets.create' as `action`, '[]' as `conditions`, 1595433593431 as `created_at`, NULL as `fields`, 18 as `id`, NULL as `properties`, NULL as `subject`, 1595433593435 as `updated_at` union all select 'plugin::upload.assets.update' as `action`, '[]' as `conditions`, 1595433593444 as `created_at`, NULL as `fields`, 19 as `id`, NULL as `properties`, NULL as `subject`, 1595433593448 as `updated_at` union all select 'plugin::upload.assets.download' as `action`, '[]' as `conditions`, 1595433593456 as `created_at`, NULL as `fields`, 20 as `id`, NULL as `properties`, NULL as `subject`, 1595433593460 as `updated_at` union all select 'plugin::upload.assets.copy-link' as `action`, '[]' as `conditions`, 1595433593469 as `created_at`, NULL as `fields`, 21 as `id`, NULL as `properties`, NULL as `subject`, 1595433593474 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '["admin::is-creator"]' as `conditions`, 1595433593646 as `created_at`, NULL as `fields`, 36 as `id`, NULL as `properties`, 'api::restaurant.restaurant' as `subject`, 1595433593651 as `updated_at` union all select 'plugin::upload.read' as `action`, '["admin::is-creator"]' as `conditions`, 1595433593671 as `created_at`, NULL as `fields`, 38 as `id`, NULL as `properties`, NULL as `subject`, 1595433593674 as `updated_at` union all select 'plugin::upload.assets.create' as `action`, '[]' as `conditions`, 1595433593681 as `created_at`, NULL as `fields`, 39 as `id`, NULL as `properties`, NULL as `subject`, 1595433593685 as `updated_at` union all select 'plugin::upload.assets.update' as `action`, '["admin::is-creator"]' as `conditions`, 1595433593691 as `created_at`, NULL as `fields`, 40 as `id`, NULL as `properties`, NULL as `subject`, 1595433593696 as `updated_at` union all select 'plugin::upload.assets.download' as `action`, '[]' as `conditions`, 1595433593703 as `created_at`, NULL as `fields`, 41 as `id`, NULL as `properties`, NULL as `subject`, 1595433593707 as `updated_at` union all select 'plugin::upload.assets.copy-link' as `action`, '[]' as `conditions`, 1595433593714 as `created_at`, NULL as `fields`, 42 as `id`, NULL as `properties`, NULL as `subject`, 1595433593718 as `updated_at` union all select 'plugin::content-manager.explorer.create' as `action`, '[]' as `conditions`, 1595433593750 as `created_at`, '["author","review"]' as `fields`, 44 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593754 as `updated_at` union all select 'plugin::content-manager.explorer.create' as `action`, '[]' as `conditions`, 1595433593774 as `created_at`, '["content","note","author","likes","restaurant"]' as `fields`, 46 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593778 as `updated_at` union all select 'plugin::content-manager.explorer.create' as `action`, '[]' as `conditions`, 1595433593786 as `created_at`, '["username","email","provider","password","resetPasswordToken","confirmed","blocked","role","reviews","likes","picture"]' as `fields`, 47 as `id`, NULL as `properties`, 'plugin::users-permissions.user' as `subject`, 1595433593790 as `updated_at` union all select 'plugin::content-manager.explorer.read' as `action`, '[]' as `conditions`, 1595433593812 as `created_at`, '["author","review"]' as `fields`, 49 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593817 as `updated_at` union all select 'plugin::content-manager.explorer.read' as `action`, '[]' as `conditions`, 1595433593836 as `created_at`, '["content","note","author","likes","restaurant"]' as `fields`, 51 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593839 as `updated_at` union all select 'plugin::content-manager.explorer.read' as `action`, '[]' as `conditions`, 1595433593847 as `created_at`, '["username","email","provider","password","resetPasswordToken","confirmed","blocked","role","reviews","likes","picture"]' as `fields`, 52 as `id`, NULL as `properties`, 'plugin::users-permissions.user' as `subject`, 1595433593852 as `updated_at` union all select 'plugin::content-manager.explorer.update' as `action`, '[]' as `conditions`, 1595433593871 as `created_at`, '["author","review"]' as `fields`, 54 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593876 as `updated_at` union all select 'plugin::content-manager.explorer.update' as `action`, '[]' as `conditions`, 1595433593896 as `created_at`, '["content","note","author","likes","restaurant"]' as `fields`, 56 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593900 as `updated_at` union all select 'plugin::content-manager.explorer.update' as `action`, '[]' as `conditions`, 1595433593908 as `created_at`, '["username","email","provider","password","resetPasswordToken","confirmed","blocked","role","reviews","likes","picture"]' as `fields`, 57 as `id`, NULL as `properties`, 'plugin::users-permissions.user' as `subject`, 1595433593913 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593920 as `created_at`, NULL as `fields`, 58 as `id`, NULL as `properties`, 'api::category.category' as `subject`, 1595433593925 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593933 as `created_at`, NULL as `fields`, 59 as `id`, NULL as `properties`, 'api::like.like' as `subject`, 1595433593937 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593946 as `created_at`, NULL as `fields`, 60 as `id`, NULL as `properties`, 'api::restaurant.restaurant' as `subject`, 1595433593950 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593958 as `created_at`, NULL as `fields`, 61 as `id`, NULL as `properties`, 'api::review.review' as `subject`, 1595433593963 as `updated_at` union all select 'plugin::content-manager.explorer.delete' as `action`, '[]' as `conditions`, 1595433593973 as `created_at`, NULL as `fields`, 62 as `id`, NULL as `properties`, 'plugin::users-permissions.user' as `subject`, 1595433593977 as `updated_at` union all select 'plugin::content-type-builder.read' as `action`, '[]' as `conditions`, 1595433593985 as `created_at`, NULL as `fields`, 63 as `id`, NULL as `properties`, NULL as `subject`, 1595433593990 as `updated_at` union all select 'plugin::upload.read' as `action`, '[]' as `conditions`, 1595433594038 as `created_at`, NULL as `fields`, 67 as `id`, NULL as `properties`, NULL as `subject`, 1595433594043 as `updated_at` union all select 'plugin::upload.assets.create' as `action`, '[]' as `conditions`, 1595433594051 as `created_at`, NULL as `fields`, 68 as `id`, NULL as `properties`, NULL as `subject`, 1595433594056 as `updated_at` union all select 'plugin::upload.assets.update' as `action`, '[]' as `conditions`, 1595433594063 as `created_at`, NULL as `fields`, 69 as `id`, NULL as `properties`, NULL as `subject`, 1595433594067 as `updated_at` union all select 'plugin::upload.assets.download' as `action`, '[]' as `conditions`, 1595433594076 as `created_at`, NULL as `fields`, 70 as `id`, NULL as `properties`, NULL as `subject`, 1595433594080 as `updated_at` union all select 'plugin::upload.assets.copy-link' as `action`, '[]' as `conditions`, 1595433594088 as `created_at`, NULL as `fields`, 71 as `id`, NULL as `properties`, NULL as `subject`, 1595433594093 as `updated_at` union all select 'plugin::upload.settings.read' as `action`, '[]' as `conditions`, 1595433594100 as `created_at`, NULL as `fields`, 72 as `id`, NULL as `properties`, NULL as `subject`, 1595433594105 as `updated_at` union all select 'plugin::content-manager.single-types.configure-view' as `action`, '[]' as `conditions`, 1595433594112 as `created_at`, NULL as `fields`, 73 as `id`, NULL as `properties`, NULL as `subject`, 1595433594116 as `updated_at` union all select 'plugin::content-manager.collection-types.configure-view' as `action`, '[]' as `conditions`, 1595433594123 as `created_at`, NULL as `fields`, 74 as `id`, NULL as `properties`, NULL as `subject`, 1595433594127 as `updated_at` union all select 'plugin::content-manager.components.configure-layout' as `action`, '[]' as `conditions`, 1595433594134 as `created_at`, NULL as `fields`, 75 as `id`, NULL as `properties`, NULL as `subject`, 1595433594138 as `updated_at` union all select 'plugin::users-permissionss.roles.create' as `action`, '[]' as `conditions`, 1595433594145 as `created_at`, NULL as `fields`, 76 as `id`, NULL as `properties`, NULL as `subject`, 1595433594149 as `updated_at` union all select 'plugin::users-permissionss.roles.read' as `action`, '[]' as `conditions`, 1595433594156 as `created_at`, NULL as `fields`, 77 as `id`, NULL as `properties`, NULL as `subject`, 1595433594160 as `updated_at` - SQLITE_ERROR: table admin_permissions has no column named fields] {
errno: 1,
code: 'SQLITE_ERROR'
}
Looking at previous issues which provide a similar error to this, people have recommended to open the target old_data.db
file and drop the "fields" column in the V3 "strapi_permission" table. Doing so will allow the migration to run its course and "successfully" complete. However, if I then try to run "npm run develop", I get this:
[2023-01-05 16:45:11.427] debug: ⛔️ Server wasn't able to start properly.
[2023-01-05 16:45:11.428] error: Cannot read properties of null (reading 'fields')
TypeError: Cannot read properties of null (reading 'fields')
at Array.<anonymous> (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\admin\server\services\content-type.js:167:21)
at Array.map (<anonymous>)
at Object.cleanPermissionFields (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\admin\server\services\content-type.js:163:22)
at Object.cleanPermissionsInDatabase (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\admin\server\services\permission\queries.js:163:26)
at async module.exports (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\admin\server\bootstrap.js:88:3)
at async Strapi.runLifecyclesFunctions (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\strapi\lib\Strapi.js:542:7)
at async Strapi.bootstrap (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\strapi\lib\Strapi.js:469:5)
at async Strapi.load (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\strapi\lib\Strapi.js:478:5)
at async Strapi.start (C:\MYOXYGEN-REPOS\PCC Backend V4\demomigration\node_modules\@strapi\strapi\lib\Strapi.js:212:9)
package.json
of the v3-sql-v4-sql
folder, you should specify "@vscode/sqlite3": "^5.1.2-vscode"
. I had to upgrade this package due to the following issue with this package: microsoft/vscode#152839.npm install
the rest of the packages in v3-sql-v4-sql
, as normal.old_data.db
.data.db
); I didn't create any admin users prior to running the script.v3-sql-v4-sql
, copy the .env.sqlite.example
file and fill out the required info, including where your v3 and v4 database file is located. Save this copied file as .env
.v3-sql-v4-sql
folder, run npm run start
.fields
column within the strapi_permission
column. Re-run the migration script with npm run start
. It should now claim it's successfully completed.npm run develop
. You should now see the second error I've pasted.The database successfully migrates, all permissions are preserved, and I can use Strapi V4 with my migrated data.
Above is a screenshot of node_modules@strapi\admin\server\services\content-type.js:167:21, where the error after "successfully migrating" occurs. I'd like to draw attention to properties: { fields }
. I'm unsure how deleting the fields
column works for other users which face a similar issue to this, as Strapi then looks for the missing column
More screenshots can be provided on request.
Not applicable, but can provide on-request.
Not applicable, but can provide on-request.
Strapi allows components to have the same name as collections but the migration script doesn't allow for this and results in link tables with only the component id set, with null
for the collection.
If a component has a one to one relationship with a collection with the same name, Strapi uses the field name inv_{collectionname}_id in the links table to distinguish the names.
e.g.
If there is a collection named activity
and a component also named activity
, Strapi will generate a table, components_category_activities_activity_links
with the fields
| id | activity_id | inv_activity_id |
I have created a PR, #67, which supports migrating into this kind of table. It's a bit of a letterbox fix but it works for my migration.
I am trying to migrate Strapi v3 with MongoDB to Strapi v3 Postgres. But I am getting the following error:
error: insert into "modules_components" ("component_id", "component_type", "field", "id", "module_id", "order") values (DEFAULT, $1, $2, $3, $4, $5) - null value in column "component_id" of relation "modules_components" violates not-null constraint
at Parser.parseErrorMessage (/home/siddhesh/Projects/cssoch/migration-scripts/v3-mongodb-v3-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/home/siddhesh/Projects/cssoch/migration-scripts/v3-mongodb-v3-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/home/siddhesh/Projects/cssoch/migration-scripts/v3-mongodb-v3-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/home/siddhesh/Projects/cssoch/migration-scripts/v3-mongodb-v3-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (node:events:527:28)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
length: 282,
severity: 'ERROR',
code: '23502',
detail: 'Failing row contains (130, Project, 1, components_project_projects, null, 38).',
hint: undefined,
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: 'public',
table: 'modules_components',
column: 'component_id',
dataType: undefined,
constraint: undefined,
file: 'execMain.c',
line: '1883',
routine: 'ExecConstraints'
}
I am not getting the cause of the error. Whether it causing because by the script or because of data.
While the scripts add data from the source to the destination, users seem to be reaching race conditions seemingly caused by PrimaryKey collisions in some database tables. The currently proposed solution from interaction with the user and observation is to add a setTimeout call between SQL create statements to slow down the process allowing the prior process to complete.
This occurred during a user's routine migration process. To our knowledge, there aren't any steps outside of what we prescribe in our guides that the users followed.
If more context is required, @kasonde or @derrickmehaffy would do their best gather the information required. For now, here's an example of the output they got from the scripts:
error: insert into "strapi_ecomm_<retracted>_v4"."admin_permissions" ("action", "conditions", "created_at", "properties", "subject", "updated_at") values ($1, $2, $3, $4, $5, $6) returning "id" - duplicate key value violates unique constraint "admin_permissions_pkey"
error: insert into "strapi_ecomm_<retracted>_v4"."admin_permissions" ("action", "conditions", "created_at", "properties", "subject", "updated_at") values ($1, $2, $3, $4, $5, $6) returning "id" - duplicate key value violates unique constraint "admin_permissions_pkey"
Anywhere you see <retracted>
is information about the user we'd prefer to keep private but has no effect on the process. It is just three letters, i.e strapi_ecomm_abc_v4
.
When running the script, the script aborts with the following error:
const v3RelationTables = tables.filter((t) => t.includes("__"));
TypeError: Cannot read properties of undefined (reading 'includes')
const v3RelationTables = tables.filter((t) => t.includes("__"));
Before starting the script, I had to adjust certain columns in the table to be compliant with the snake_case.
Relation fields on the user model fails to migrate to V4 as schema is not loaded in migrateUser.js.
[Error: insert into up_users
(all insert queries), SQLITE_ERROR: table up_users has no column named country] {
errno: 1,
code: 'SQLITE_ERROR'
}
The script should create relations on the user model in the same way as custom made models.
This issue is only impactful because:
The properties field in 'strapi_permission' is not-required, and has a default value of {}, so realistically everyone should have this in their MongoDB. However, some may have migrated from older versions, and only have the 'fields' property. This could also silently cause other issues with long-time mongo databases being migrated
When an attribute that has a default value is undefined, and the migration assistant copies it to the SQL, it will be copied as NULL.
Inside the SQL, there is no 'DEFAULT' specified for columns that have a 'default' value in the core_store entry
SELECT * FROM strapi_permission WHERE properties IS NULL;
yarn install && yarn develop
in your app directoryEntries which have default values in core_store, will have their default values inserted into the SQL if it is otherwise left NULL during insertion.
Strapi_permission core_store structure - Note: relevant fields are 'attributes>properties|conditions', which has 'default: {}'
"uid":"strapi::permission",
"collectionName":"strapi_permission",
"kind":"collectionType",
"info":{
"name":"Permission",
"description":""
},
"options":{
"timestamps":[
"createdAt",
"updatedAt"
]
},
"pluginOptions":{
"content-manager":{
"visible":false
},
"content-type-builder":{
"visible":false
}
},
"attributes":{
"action":{
"type":"string",
"minLength":1,
"configurable":false,
"required":true
},
"subject":{
"type":"string",
"minLength":1,
"configurable":false,
"required":false
},
"properties":{
"type":"json",
"configurable":false,
"required":false,
"default":{
}
},
"conditions":{
"type":"json",
"configurable":false,
"required":false,
"default":[
]
},
"role":{
"configurable":false,
"model":"role",
"plugin":"admin"
}
}
}
Note: This error only applies to long-term mongoDB users who's data has persisted long enough to not have a 'properties' field in strapi_permissions/have the old 'fields' column, and are attempting to migrate. This issue is for when an attribute does not have a 'non-required' attribute specified (that has a default value)
For long-time mongoDB users, they might have production databases without properties, or have the old 'fields' attribute.
When the properties field in mongoDB is undefined, it is migrated as null to SQL. Null values properties cause 'Type Error: Cannot convert undefined or null to object' error in /Path/To/Strapi/app/node_modules/strapi-admin/services/role.js:51:17
Running v3-sql-v4-sql migration, got an error on table contentName_localizations migration.
For some data models : no error but invalid results (only contentName_id column is populated)
For some data models : get error : column "contentName_id" of relation "contentName_localizations_links" does not exist
yarn start
Should migrate localizations without errors
Migrating 2 items from companies__localizations to companies_localizations_links
companies__localizations batch #1
/scripts/strapi_migrations/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287
const message = name === 'notice' ? new messages_1.NoticeMessage(length, messageValue) : new messages_1.DatabaseError(messageValue, length, name);
^
error: insert into "companies_localizations_links" ("company_id", "related_company_id") values ($1, $2), ($3, $4) - column "related_company_id" of relation "companies_localizations_links" does not exist
at Parser.parseErrorMessage (/scripts/strapi_migrations/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/scripts/strapi_migrations/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/scripts/strapi_migrations/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.<anonymous> (/scripts/strapi_migrations/migration-scripts/v3-sql-v4-sql/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (node:events:390:28)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at Socket.Readable.push (node:internal/streams/readable:228:10)
at TCP.onStreamRead (node:internal/stream_base_commons:199:23) {
length: 160,
severity: 'ERROR',
code: '42703',
detail: undefined,
hint: undefined,
position: '60',
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'parse_target.c',
line: '1066',
routine: 'checkInsertTargets'
}
error Command failed with exit code 1.
v3-sql-v4-sql (pg 13 to pg13)
3.6.8 -> 4.1.9
Thanks for the migration scripts. However when executing the migration, I get the error:
Column »author« of relation »components_blog_blog_bases« does not exists.
.
In the mentioned v3 component we have a OneToOne relation to author like this (as described here for content types):
"author": {
"model": "author"
}
We migrated code to v4 like this:
"author": {
"type": "relation",
"relation": "oneToOne",
"target": "api::author.author"
}
Strapi creates the following tables in database (new table components_blog_blog_bases_categories_links on v4):
DBv3:
components_blog_blog_bases
components_blog_blog_bases__categories
DBv4:
components_blog_blog_bases
components_blog_blog_bases_author_links
components_blog_blog_bases_categories_links
Is it correct that strapi v4 creates the new table components_blog_blog_bases_categories_links for OneToOne relations?
It looks like, that you implemented this for models but not for components?
Can you please provide a solution to migrate relations in components?
Simple Component with a 1:1 Relation.
A 1:1 Relation gets added as a separate column with the id of the target.
A 1:Many Releation (here categories) is a separate table
V4 creates separate tables for 1:1 and 1:Many Relations
The script is however still looking for the author column in the v4 table, which of course now does not exist.
yarn 3.2.0
node 16.15.0
postgres 13
`const message = name === 'notice' ? new messages_1.NoticeMessage(length, messageValue) : new messages_1.DatabaseError(messageValue, length, name);
^
error: insert into "up_users" (...) - column "name" of relation "up_users" does not exist`
I run data migration from 3.6.10 version to latest 4.5.1. The migration finished, but for localized entries no links were correctly created. The _localization_links
tables are not empty, but the _inv
field is always null. Like:
Media entries in the upload_file table do get copied into the new files table but are not displayed in the strapi media library ui (I'm upgrading to strapi version 4.12.4). I found this could be fixed by adding folder_path: "/"
below line 44 in https://github.com/strapi/migration-scripts/blob/30aedca3233ae28475dba301b963b4f106c90119/v3-sql-v4-sql/migrate/migrateFiles.js
Unfortunately I don't have the capacity to write a proper PR and do testing and all that but I thought I should open an issue in case anyone else runs into this and it's helpful to them or someone wants to properly fix this.
Ta!
When running the script, if a column with a many-to-many relation is empty or contains null values, the script terminates prematurely. This issue occurs because the map
method is called on a null value in such cases.
The script should gracefully handle cases where columns in many-to-many relations are empty or contain null values, allowing it to continue processing other data.
Found a solution for my specific use case adding guards like this
line 214 : const rows = (entry[key] ?? []).map((e, idx) => ({ ... })
line 270 : if (!entry[key] || !idMap.get(entry[key])) { continue; }
I don't know if i shoud submit a Pull Request with this fix or if something more robust will be added, let me know !
migrate
In some cases during migration, users core_store gets corrupted causing an inability to view collection types in the content manager. This currently only occurred for a collection type and not a single type.
This occurred during a user's routine migration process. To our knowledge, there aren't any steps outside of what we prescribe in our guides that the users followed.
To get past this, the user had to delete their core_store content manager config
Thanks so much for making these scripts!
v3 mongo=> v3 sql timestamps aren't converting for me.
I find it strange in the code that:
Also, the if
isn't working for me when i do change around these lines, since here:
key
will be created_at
for me and model.options.timestamps
is [ 'createdAt', 'updatedAt' ]
opposed to the code assuming model.options.timestamps[0] === "created_at"
(also it is strange both
and
are looking for the [0] index.
I'll make a PR after this report :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.