Upgrade from 5.0.3 to 5.1.0

any problems during the update? How was it applied?

Could you please provide the content of the #__tuf_metadata table?

from joomla-cms.

Where can I find information on these changes to remedy them on my hosting?

Joomla has implemented TUF as its updating framework https://theupdateframework.io/

We are using PHP-TUF as the implementation within Joomla to consume the updates: https://github.com/joomla-backports/php-tuf

And PHP-TUF uses the compat package for lib_sodium to do the crypto operations: https://github.com/paragonie/sodium_compat

The error message occurs in the last part of the chain, the crypto library.

What I need to check and implement on my hosting to avoid these problems?

That's a though one. At a first step, check if your PHP native sodium support: https://www.php.net/manual/en/sodium.installation.php and if not try to enable it. If that solves the issue, it's most likely a bug in the upstream library that needs further debugging.

from joomla-cms.

Interesting. Can you at least provide the used PHP version and operation system so I can have same basic information to look for the root cause in the upstream library?

Windows system
Apache 2.4
PHP 8.2
Custom config

from joomla-cms.

Disabling the Update Sites >>> Joomla! Core plugin bring back to green all 3 buttons (home dashboard).

This allow searching and updating installed extensions (with this warning message "Some update sites are disabled. You may want to check the Update Sites Manager") but if you check for Joomla update it automatically enable the "Joomla! Core plugin" and shows this warning:
(success) Checked for updates.
An error has occurred.
0 Could not load root metadata.

Returning to Dashboars then shows the 3 buttons with red background.

from joomla-cms.

Please paste the contents of the update log

from joomla-cms.

Update completed successfully with the internal wizard / guided procedure.

I've done that in several other sites without issues (same extensions installed).

#__tuf_metadata table was empty.

I cloned it from a similar installation and everything worked!
It was easy after all, thank you very much!

from joomla-cms.

This means that the update was NOT successful. Please pust the update log

from joomla-cms.

This is the content of the "/administrator/logs/joomla_update.php" file.

#Date: 2024-04-17 06:32:52 UTC
#Software: Joomla! 5.0.3 Stable [ Kuboresha ] 20-February-2024 16:00 GMT

#Fields: datetime priority clientip category message
2024-04-17T06:32:52+00:00 INFO 195.210.40.226 update Test logging
2024-04-17T06:32:52+00:00 INFO 195.210.40.226 update Update started by user alecrespi (223). Old version is 5.0.3.
2024-04-17T06:32:53+00:00 INFO 195.210.40.226 update Downloading update file from https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/joomla5/Joomla_5.1.0-Stable-Update_Package.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA6LXDJLNUINX2AVMH%2F20240417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240417T063252Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=4103a267307c44ecffb07f9d49269937adf505822c043317dfeeb2faf9f745bd.
2024-04-17T06:32:57+00:00 INFO 195.210.40.226 update File Joomla_5.1.0-Stable-Update_Package.zip downloaded.
2024-04-17T06:32:58+00:00 INFO 195.210.40.226 update Starting installation of new version.
2024-04-17T09:32:53+00:00 INFO 195.210.40.226 update Test logging
2024-04-17T09:32:53+00:00 INFO 195.210.40.226 update Uploading update file
2024-04-17T09:33:57+00:00 INFO 195.210.40.226 update File [ROOT][TMP]/juI32R3Y downloaded.
2024-04-17T09:33:58+00:00 INFO 195.210.40.226 update Starting installation of new version.
2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Finalising installation.
2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Start of SQL updates.
2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update The current database version (schema) is 5.1.0-2024-03-28.
2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update End of SQL updates.
2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Uninstalling extensions
2024-04-17T09:34:54+00:00 INFO 195.210.40.226 update Deleting removed files and folders.
2024-04-17T09:34:57+00:00 INFO 195.210.40.226 update Cleaning up after installation.
2024-04-17T09:34:57+00:00 INFO 195.210.40.226 update Update to version 5.1.0 is complete.

from joomla-cms.

That explains why the TUF database did not contain anything and it also shows that you do not have a completed update,

No SQL updates took place!!

This is what I would have expected to see. @richard67 any thoughts?

#Date: 2024-04-17 10:09:35 UTC
#Software: Joomla! 5.0.3 Stable [ Kuboresha ] 20-February-2024 16:00 GMT

from joomla-cms.

To be precise, I performed a second update with the "Upload and Update" function... perhaps this log is redirected on the second attempt?
Is there a way to recover the previous log?

At the end of the first attempt (online update) in the System >> Maintenance >> Database section there were inconsistencies on the "Joomla CMS" line and I used the "Update Structure" button.
After that everything seemed ok (except for the errors in my initial post).

from joomla-cms.

yes that would explain why the log shows no updates.

"Update Structure" does exactly AND only that. It will not make other changes that may be needed like additonal data records. So it would create a missing tuf database table but it would not create the missing data in that table,

from joomla-cms.

So that would mean that the first update most likely failed before the SQL statements could be executed. The repair process created the table but not the row, resulting in the missing data.

from joomla-cms.

If only the #__tuf_metadata table was involved, can I consider the issue closed?
Or do I need to check some other table?

from joomla-cms.

If only the #__tuf_metadata table was involved, can I consider the issue closed? Or do I need to check some other table?

@alecrespi78 If it is right that no SQL statements were executed with the first update, then all script named like 5.1.0-*.sql which could be found here have not run: https://github.com/joomla/joomla-cms/tree/5.1-dev/administrator/components/com_admin/sql/updates/mysql .

These are the scripts

• administrator/components/com_admin/sql/updates/mysql/5.1.0-2023-11-28.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-01-04.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-10.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-24.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-25.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-03-08.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-03-28.sql

The "Fix structure" button has only run the DDL (data definition language) SQL statements, i.e. "CREATE TABLE" or "ALTER TABLE", but not the DML (data manipulation language) SQL statements like e.g. "INSERT" or "UPDATE" or "DELETE". You should check in phpMyAdmin if it needs to run these statements (i.e. check if the result of that statement is already there or not) and have to run them in phpMyAdmin.

from joomla-cms.

I have the same problem after update to 5.1.
The joomla update log not contain errors but I get "0 Internal check failed." in Extensions Update and more.

An error has occurred.

0 Internal check failed.

I have this issue on all updated sites so I hope in a quick update to fix this issue.

from joomla-cms.

please post the update log and also the method you used to perform the update!!

from joomla-cms.

Hi Brian, I updated Joomla by the automated Joomla Backend functionality and this is the update log:

#
#Date: 2024-04-17 19:22:02 UTC
#Software: Joomla! 5.0.3 Stable [ Kuboresha ] 20-February-2024 16:00 GMT

#Fields: datetime priority clientip category message
2024-04-17T19:22:02+00:00 INFO 213.203.154.248 update Test logging
2024-04-17T19:22:02+00:00 INFO 213.203.154.248 update Update started by user Umberto Celentano (797). Old version is 5.0.3.
2024-04-17T19:22:03+00:00 INFO 213.203.154.248 update Downloading update file from https://s3-us-west-2.amazonaws.com/joomla-official-downloads/joomladownloads/joomla5/Joomla_5.1.0-Stable-Update_Package.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA6LXDJLNUINX2AVMH%2F20240417%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240417T192208Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=626da0ae032c6fe4d8d3827b030dcede9a03d53b8e6a0a20617cd4ab36683a10.
2024-04-17T19:22:07+00:00 INFO 213.203.154.248 update File Joomla_5.1.0-Stable-Update_Package.zip downloaded.
2024-04-17T19:22:07+00:00 INFO 213.203.154.248 update Starting installation of new version.
2024-04-17T19:22:47+00:00 INFO 213.203.154.248 update Finalising installation.
2024-04-17T19:22:47+00:00 INFO 213.203.154.248 update Start of SQL updates.
2024-04-17T19:22:47+00:00 INFO 213.203.154.248 update The current database version (schema) is 5.0.0-2023-09-11.
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2023-11-28. Query text: INSERT INTO #__extensions (package_id, name, type, element, folder, .
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-01-04. Query text: ALTER TABLE #__fields_values MODIFY value MEDIUMTEXT;.
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-10. Query text: ALTER TABLE #__guidedtour_steps ADD COLUMN params text NULL ;.
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-24. Query text: CREATE TABLE IF NOT EXISTS #__tuf_metadata ( id int NOT NULL AUTO_INCREMEN.
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-24. Query text: INSERT INTO #__tuf_metadata (update_site_id, root) SELECT ue.update_site_. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-24. Query text: UPDATE #__update_sites SETtype= 'tuf',location= 'https://update.joo. 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-02-25. Query text: INSERT INTO#__extensions (package_id, name, type, element, folder, . 2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: INSERT IGNORE INTO #__guidedtour_steps (tour_id, title, published, desc.
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: ALTER TABLE #__guidedtours ADD COLUMN autostart int NOT NULL DEFAULT 0 ;.
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: INSERT IGNORE INTO #__guidedtours (title, uid, description, ordering, .
2024-04-17T19:22:48+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-08. Query text: UPDATE #__guidedtour_steps SET tour_id = LAST_INSERT_ID() WHERE tour_id=0;.
2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update Ran query from file 5.1.0-2024-03-28. Query text: INSERT INTO #__postinstall_messages (extension_id, title_key, `description.
2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update End of SQL updates.
2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update Uninstalling extensions
2024-04-17T19:22:49+00:00 INFO 213.203.154.248 update Deleting removed files and folders.
2024-04-17T19:22:51+00:00 INFO 213.203.154.248 update Cleaning up after installation.
2024-04-17T19:22:51+00:00 INFO 213.203.154.248 update Update to version 5.1.0 is complete.

from joomla-cms.

@umbcel

I have the same problem after update to 5.1.

No, it’s definitely not the same issue. You get a completely different error message compared to the one in the initial post of this issue.

Yours:

0 Internal check failed.

The one from this issue:

0 Could not load root metadata.

And your log shows that all SQL statements were run without error.

So maybe you should open a new issue for your case.

from joomla-cms.

If only the #__tuf_metadata table was involved, can I consider the issue closed? Or do I need to check some other table?

@alecrespi78 If it is right that no SQL statements were executed with the first update, then all script named like 5.1.0-*.sql which could be found here have not run: https://github.com/joomla/joomla-cms/tree/5.1-dev/administrator/components/com_admin/sql/updates/mysql .

These are the scripts

• administrator/components/com_admin/sql/updates/mysql/5.1.0-2023-11-28.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-01-04.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-10.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-24.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-02-25.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-03-08.sql
• administrator/components/com_admin/sql/updates/mysql/5.1.0-2024-03-28.sql

The "Fix structure" button has only run the DDL (data definition language) SQL statements, i.e. "CREATE TABLE" or "ALTER TABLE", but not the DML (data manipulation language) SQL statements like e.g. "INSERT" or "UPDATE" or "DELETE". You should check in phpMyAdmin if it needs to run these statements (i.e. check if the result of that statement is already there or not) and have to run them in phpMyAdmin.

Thank you,
I applied them all!

from joomla-cms.

I have this issue on all updated sites so I hope in a quick update to fix this issue.

@umbcel are all those sites hosted on the same machine or at the same hosting company? Can you post the content of the tuf_metadata table?

from joomla-cms.

I have this issue on all updated sites so I hope in a quick update to fix this issue.

@umbcel are all those sites hosted on the same machine or at the same hosting company? Can you post the content of the tuf_metadata table?

Yes, all those sites are hosted on the same machine.
The contemt of the tuf_metadata table is:
1 1 {"signed":{"_type":"root","spec_version":"1.0","version":2,"expires":"2025-03-02T11:22:17Z","keys":{"07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"9b2af2d9b9727227735253d795bd27ea8f0e294a5f3603e822dc5052b44802b9"}},"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"a18e5ebabc19d5d5984b601a292ece61ba3662ab2d071dc520da5bd4f8948799"}},"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"cb0a7a131961a20edea051d6dc2b091fb650bd399bd8514adb67b3c60db9f8f9"}},"31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"589d029a68b470deff1ca16dbf3eea6b5b3fcba0ae7bb52c468abc7fb058b2a2"}},"9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc":{"keytype":"ed25519","scheme":"ed25519","keyid_hash_algorithms":["sha256","sha512"],"keyval":{"public":"6043c8bacc76ac5c9750f45454dd865c6ca1fc57d69e14cc192cfd420f6a66a9"}}},"roles":{"root":{"keyids":["1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"snapshot":{"keyids":["07eb082f367c034a95878687f6648aa76d93652b6ee73e58817053d89af6c44f","2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e"],"threshold":1},"targets":{"keyids":["31dd7c7290d664c9b88c0dead2697175293ea7df81b7f24153a37370fd3901c3"],"threshold":1},"timestamp":{"keyids":["9e41a9d62d94c6a1c8a304f62c5bd72d84a9f286f27e8327cedeacb09e5156cc"],"threshold":1}},"consistent_snapshot":true},"signatures":[{"keyid":"2dcaf3d0e552f150792f7c636d45429246dcfa34ac35b46a44f5c87cd17d457e","sig":"2a225a560ec0837b721d4c5e379fedbd3c7c9079a94e6b31e47e0184c8b95421b6036b4286c5d90f29ab4c468d79a712fdb65e96511394ceb3aa8e2b3983a501"},{"keyid":"1b1b1dd55b2c1c7258714cf1c1ae06f23e4607b28c762d016a9d81c48ffe5669","sig":"8ce0b2a7bdc1e6dcba12081f440510df0a593c072dcf591631c2dd0f456844a7da63be8e8ac31ffbddf42641fde84dc733a336031d182c2163b4c1eaf2117005"}]}

from joomla-cms.

@umbcel can you please try to reproduce the issue with the same site but on a different machine at a different host? The error seems to be related to the handling of the cryto logic in an upstream library and I assume that some sort of environment issue (OS, PHP Version etc) plays a role. Would like to confirm that assumption.

from joomla-cms.

@umbcel can you please try to reproduce the issue with the same site but on a different machine at a different host? The error seems to be related to the handling of the cryto logic in an upstream library and I assume that some sort of environment issue (OS, PHP Version etc) plays a role. Would like to confirm that assumption.

I can't. I have all those web sites on my machine, my hosting, and no problem happen with older 3.x/4.x/5.x Joomla releases. This problem started after I updated web sites to Joomla 5.1.

from joomla-cms.

Well that's a pity. Could you please at least provide as many environment informations as possible? OS, PHP info() output, CPU?

from joomla-cms.

Well that's a pity. Could you please at least provide as many environment informations as possible? OS, PHP info() output, CPU?

I prefer not share system info in public environment for security reason. I tried also to install a fresh Joomla 5.1 and the problem happen again.

from joomla-cms.

I tried also to install a fresh Joomla 5.1 and the problem happen again.

... which supports the assumption that the problem is related to the environment.

from joomla-cms.

I tried also to install a fresh Joomla 5.1 and the problem happen again.

... which supports the assumption that the problem is related to the environment.

So why was everything working fine just before installing the updates? Why did the previous version of Joomla 5 work well?

from joomla-cms.

So why was everything working fine just before installing the updates? Why did the previous version of Joomla 5 work well?

Because Joomla 5.1 introduced a new function that ensures that an update, that is fetched from the joomla server, is actually a legit one and not one that has been manipulated by an attacker.

The error message is having its source in the cryptographic verification process which works fine on a wide range of different machines. In fact, you are the only person with that issue so far - so something about that environment has to be special.

from joomla-cms.

So why was everything working fine just before installing the updates? Why did the previous version of Joomla 5 work well?

Because Joomla 5.1 introduced a new function that ensures that an update, that is fetched from the joomla server, is actually a legit one and not one that has been manipulated by an attacker.

The error message is having it's source in the cryptographic verification process which works fine on a wide range of different machines. In fact, you are the only person with that issue so far - so something about that environment has to be special.

Ok, thanks, this is a more logical explanation and is likely to be the case. Where can I find information on these changes to remedy them on my hosting?

from joomla-cms.

What I need to check and implement on my hosting to avoid these problems?

from joomla-cms.

Yeahhhh! Problem solved! I just enabled Sodium support in PHP and all run fine now.
Thank you very much!

from joomla-cms.

Interesting. Can you at least provide the used PHP version and operation system so I can have same basic information to look for the root cause in the upstream library?

from joomla-cms.

I'm getting these errors as well. My environment is Apache 2.4, PHP 8.3.6. I'm running Joomla! 5.1.0. Here's the stack trace:

2024-05-19T20:45:45+00:00 CRITICAL 45.17.166.59 error Uncaught Throwable of type JsonException thrown with message "Syntax error". Stack trace: #0 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Metadata/MetadataBase.php(90): json_decode('', true, 512, 4194304)
akeeba/panopticon#1 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Repository.php(67): Tuf\Metadata\MetadataBase::createFromJson('')
akeeba/panopticon#2 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Updater.php(181): Tuf\Client\Repository->getTimestamp()
akeeba/panopticon#3 [ROOT]/libraries/vendor/php-tuf/php-tuf/src/Client/Updater.php(148): Tuf\Client\Updater->updateTimestamp()
akeeba/panopticon#4 [ROOT]/libraries/src/TUF/TufFetcher.php(117): Tuf\Client\Updater->refresh()
akeeba/panopticon#5 [ROOT]/libraries/src/Updater/Adapter/TufAdapter.php(84): Joomla\CMS\TUF\TufFetcher->getValidUpdate()
akeeba/panopticon#6 [ROOT]/libraries/src/Updater/Adapter/TufAdapter.php(50): Joomla\CMS\Updater\Adapter\TufAdapter->getUpdateTargets(Array)
akeeba/panopticon#7 [ROOT]/libraries/src/Updater/Updater.php(263): Joomla\CMS\Updater\Adapter\TufAdapter->findUpdate(Array)
akeeba/panopticon#8 [ROOT]/libraries/src/Updater/Updater.php(164): Joomla\CMS\Updater\Updater->getUpdateObjectsForSite(Array, 4, false)
akeeba/panopticon#9 [ROOT]/administrator/components/com_installer/src/Model/UpdateModel.php(275): Joomla\CMS\Updater\Updater->findUpdates(0, 3600, 4)
akeeba/panopticon#10 [ROOT]/administrator/components/com_installer/src/Controller/UpdateController.php(161): Joomla\Component\Installer\Administrator\Model\UpdateModel->findUpdates(0, 3600, 4)
akeeba/panopticon#11 [ROOT]/libraries/src/MVC/Controller/BaseController.php(730): Joomla\Component\Installer\Administrator\Controller\UpdateController->ajax()
akeeba/panopticon#12 [ROOT]/libraries/src/Dispatcher/ComponentDispatcher.php(143): Joomla\CMS\MVC\Controller\BaseController->execute('ajax')
akeeba/panopticon#13 [ROOT]/libraries/src/Component/ComponentHelper.php(361): Joomla\CMS\Dispatcher\ComponentDispatcher->dispatch()
akeeba/panopticon#14 [ROOT]/libraries/src/Application/AdministratorApplication.php(150): Joomla\CMS\Component\ComponentHelper::renderComponent('com_installer')
akeeba/panopticon#15 [ROOT]/libraries/src/Application/AdministratorApplication.php(195): Joomla\CMS\Application\AdministratorApplication->dispatch()
akeeba/panopticon#16 [ROOT]/libraries/src/Application/CMSApplication.php(306): Joomla\CMS\Application\AdministratorApplication->doExecute()
akeeba/panopticon#17 [ROOT]/administrator/includes/app.php(58): Joomla\CMS\Application\CMSApplication->execute()
akeeba/panopticon#18 [ROOT]/administrator/index.php(32): require_once('/home/amybitcov...')
akeeba/panopticon#19 {main}

from joomla-cms.

I have the same error. I clicked on it and the 5.1.1 update appeared. After that I get a Could not load root metadata error. After that I just get a red X to update. I think this error should be handled if the tuf_metadata table is empty somewhere.

from joomla-cms.