Giter Club home page Giter Club logo

Comments (8)

jbuchman avatar jbuchman commented on August 26, 2024

I also tested v0.29.0, v0.28.0, and v0.27.2

None of these worked as expected.

However, v0.27.1 does work as expected.

buchmanj@Accio ~/D/sqleet-v0.27.1> ./sqleet
SQLite version 3.27.1 2019-02-08 13:17:39
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> ^D
buchmanj@Accio ~/D/sqleet-v0.27.1> ./sqleet test.sqlite
SQLite version 3.27.1 2019-02-08 13:17:39
Enter ".help" for usage hints.
sqlite> PRAGMA rekey='swordfish';
sqlite> ^D
buchmanj@Accio ~/D/sqleet-v0.27.1> hexdump -C test.sqlite
00000000  14 c2 ed e3 36 fb f2 9c  27 75 7d 9d f8 6b 61 0a  |....6...'u}..ka.|
00000010  56 61 e3 62 7c 76 8f 30  12 54 74 d0 7b c4 fd 1e  |Va.b|v.0.Tt.{...|
00000020  f2 26 f3 c7 80 f3 8c 23  4f a5 ef e3 d4 8e 03 a7  |.&.....#O.......|
00000030  e8 c6 cf de 96 2c 33 70  2e 0b 5e b1 22 50 63 e9  |.....,3p..^."Pc.|
00000040  66 00 47 72 d9 39 e8 9a  38 a6 7d b7 4c 07 5e ad  |f.Gr.9..8.}.L.^.|
00000050  f4 b8 50 4e 5c 0d 21 d8  03 a2 82 c4 6e b3 1b 36  |..PN\.!.....n..6|
00000060  c6 76 52 fc 1f 84 c9 3e  2b 7b 55 cc 43 8f 13 d3  |.vR....>+{U.C...|
00000070  bc 41 8f 53 36 96 35 91  47 30 c3 78 ed c8 5d f7  |.A.S6.5.G0.x..].|
00000080  05 b2 82 84 55 cf 98 25  a3 32 5c aa f9 9f 36 dc  |....U..%.2\...6.|
00000090  c5 e6 8b 29 0d cb aa db  e8 1d d3 27 1c e6 11 af  |...).......'....|
000000a0  34 5c c7 04 7c f3 63 f5  ad 5b b5 5b 51 52 d8 59  |4\..|.c..[.[QR.Y|

from sqleet.

jbuchman avatar jbuchman commented on August 26, 2024

I've tested this on both Mac, BSD and Linux based systems. It seems that it's only failing on Mac / iOS systems. I was not able to reproduce on a BSD or Linux box. It only fails on Mac/iOS systems.

from sqleet.

nyneplus avatar nyneplus commented on August 26, 2024

I can confirm the rekey issue. Android jni and windows both do not work. Only 0.27.1 does.

from sqleet.

resilar avatar resilar commented on August 26, 2024

Sorry for the delay, especially given the serious nature of the bug. Automated cross-platform tests are planned to be implemented prior to v1 in order to catch platform-specific issues like this in the future.

TL;DR: Fix in master. Please test and report back as I have yet to reproduce this issue myself. A fixed release sqleet v0.30.1 will be published once the fix is confirmed to resolve the issue.


I do not have Windows at hand for reproducing, but AddressSanitizer❤️ found an error introduced with URI API (shortly after 0.27.1) which likely leads to rekey failing in certain conditions. Rekey attaches a vacuum database that has no filename (URI) associated with it, which results in sqleet trying to parse URI configuration from an empty string zUri.

145 int codec_parse_uri_config(Codec *codec, const char *zUri)
146 {
147     int rc, pagesize;
148     const char *value;
149 
150     /* Override page_size PRAGMA */
151     pagesize = (int)sqlite3_uri_int64(zUri, "page_size", -1);

Line 151 calls sqlite3_uri_int64() with a zero-length URI, that is, zUri[0] = '\0'. Internally, zUri gets passed to sqlite3_uri_parameter() for parsing the value of the page_size URI parameter:

160417 SQLITE_API const char *sqlite3_uri_parameter(const char *zFilename, const char *zParam){
160418   if( zFilename==0 || zParam==0 ) return 0;
160419   zFilename += sqlite3Strlen30(zFilename) + 1;
160420   while( zFilename[0] ){

Upon entering the function, zFilename (i.e., the passed-in zUri) is a pointer to an empty string. Line 160419 increments the pointer by 1, thereby causing an out-of-bound access (zUri[1]) in the while condition in line 160420.

The fix is trivial: Set zUri to NULL if codec_parse_uri_config() receives an empty URL string. Interestingly, (internal) SQLite3 URI functions handle NULL URIs correctly, whereas empty URI strings produce undefined behavior.

Fixed in master commit 695bc40.

from sqleet.

resilar avatar resilar commented on August 26, 2024

I was unable to reproduce the rekeying error using Win10 and msys2 gcc. Probably depends on the compiler used in addition to operating system. This is not surprising assuming the root cause is an out-of-bound memory access in heap (such bugs tend to be nondeterministic in nature).

However, the fact that the bug seems to be introduced between releases v0.27.1 and v0.27.2 makes me slightly worried. URI API was added in v0.30.0 so the bug described in my previous comment did not exist before v0.30.0. There also were no significant changes in sqleet or SQLite3 code between vesions v0.27.1 and v0.27.2 that could explain the rekey bug.

Nonetheless, let's see if the proposed fix addresses the rekey bug. If not, I will try to figure out some way to reproduce the issue in my test environment.


Must be noted that SQLITE_MISUSE is a very rare error code in SQLite3 code base and sqleet returns it only in the case of invalid URI configuration. Thus, SQLITE_MISUSE strongly suggests a problem in URI configuration parsing. Rekeying being broken before the introduction of URI API (v0.30.0) is unexpected.

from sqleet.

resilar avatar resilar commented on August 26, 2024

No luck reproducing on Android in Termux environment with clang-compiled sqleet v0.30.0 (unfixed release). Oh well, hopefully someone who has been able to trigger the bug can test the fix in latest master branch.

from sqleet.

nyneplus avatar nyneplus commented on August 26, 2024

I tested the fix on Android JNI 21 And it seems to work perfectly. Windows i can test but only POSIX compiled like clang/gcc.

from sqleet.

resilar avatar resilar commented on August 26, 2024

That's great! Fixed sqleet v0.30.1 release is out now. I guess I have to figure out a systematic way to backport these kind of critical fixes to older releases as well.

If it turns out that the fix is incomplete and rekeying still fails in other environments, feel free to reopen this issue.

Thanks everybody for your contribution to sqleet.

from sqleet.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.