Below is the code that calculates the progress of an ongoing headers fetch operation.
https://github.com/raedahgroup/dcrlibwallet/blob/d4d67d9b370539a6c0557685bf2319cecaecf5b4/syncnotification.go#L107-L110

The calculation done here is potentially flawed because fetchedHeadersCount which really is lastFetchedHeaderHeight may not be the total number of headers fetched so far in THIS sync operation. It may include headers previously fetched.

Recommendation is to determine the actual number of headers fetched so far in THIS sync operation and compare that number against the estimated number of headers left to fetch in the sync operation in order to determine the progress so far.

This calculation error may be the reason for the anomaly previously noted in the dcrlibwallet source code:
https://github.com/raedahgroup/dcrlibwallet/blob/d4d67d9b370539a6c0557685bf2319cecaecf5b4/syncnotification.go#L124-L129

Follow up to #57.

It's been considered to use the same encryption process dcrwallet uses for encrypting extended private keys before saving to db. This method has been reported to previously cause out-of-memory crashes on mobile devices, so alternative encryption algorithms could be considered.

It's currently possible to add 2 wallets with the same seed to the app, which makes txs appear double on the recent transactions list .

Should be possible to prevent adding a wallet if it uses the same seed as a previously added wallet when decred/dcrwallet#1037 is fixed.

It also fails when given an empty string as rootDir.

Follow up to #57.

It can be expected that the structure of data saved to the wallets.db database introduced in #57 may change over time, as features get added, removed or modified.

We'll need a db versioning system with provisions for upgrading the wallets database from a lower version to a higher version, similar to what exists in dcrwallet.

Uncovered sync-related issues are added as comments below with updates provided as the issues are partly or completely resolved. This issue will remain open for some time to enable tracking issues with and improvements to the sync progress estimation algorithm.

Wallet creation fails on android devices that has no support for unix flock system call with the following error: loader.CreateNewWallet:: wallet.CreateDB: Cannot acquire directory lock on "/data/user/0/com.decred.dcrandroid.testnet/files/wallet/testnet3/wallet.db". Another process is using this Badger database.: permission denied. See: planetdecred/dcrandroid#352

Additional Information
Issue seems to occur on devices running Android 6 only.
Error was reproduced on Infinix hot 4 running Android 6.0 and Samsung Galaxy s5 running 6.0.1

Logs from Adb shell logcat:
08-21 10:26:26.738 2819 2819 E audit : type=1400 msg=audit(1566379586.728:1930): avc: denied { lock } for pid=14095 comm="Thread-21419" path="/data/data/com.decred.dcrandroid.testnet/files/wallet/testnet3/wallet.db" dev="mmcblk0p26" ino=360557 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:app_data_file:s0:c512,c768 tclass=dir permissive=0

Log suggest that SELinux is blocking the flock call, and issue seems to be a known bug in android 6. See this.

Transferred from planetdecred/dcrandroid#222
Bolt db crashes when wallet database size is getting close to 250mb. At that point, it's unable to open wallet because the allocated app memory cannot handle that. The crash usually starts from when the wallet is fetching headers and the wallet db rises to the size which the app's allocated memory cannot handle.

A function for fetching the proposal details of a proposal using the censorship token is required

This is a reference to the API Docs https://github.com/decred/politeia/blob/master/politeiawww/api/www/v1/api.md#proposal-details

The preferred response for this would be a blocking response

These logs show up when fetching headers, could have something to do with the current wallet version used in master

2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;
2019-04-06 19:08:19.008 [WRN] WLLT: Old vote version detected (v6), please update your wallet to the latest version.;

The amount field in the Transaction struct returns the fee for purchasing a ticket instead of the actual amount until the ticket is voted.

How to reproduce? Give NewMultiWallet invalid arguments and observe what it returns.

Why? Because the code wrongly uses errors.E. This returns just netType.

Crash usually happens when a mobile app (with generally lower memory) attempts to create or open more wallets than the device memory can handle.

Crash has also been reported to happen on an app with just one wallet (900+ transactions) when unlocking the wallet for accounts discovery.

The following truncated crash log reveals the issue to be associated with attempted memory allocation in dcrwallet's secret key derivation method.

2020-01-28 07:43:51.447 com.decred.dcrandroid.testnet E/Go: runtime: out of memory: cannot allocate 268435456-byte block (1336705024 in use)
2020-01-28 07:43:51.447 com.decred.dcrandroid.testnet E/Go: fatal error: out of memory
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime stack:
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.throw(0x8a275352, 0xd)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/panic.go:774 +0x70
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.largeAlloc(0x10000000, 0x6d4f0101, 0xebaf3730)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/malloc.go:1140 +0x11a
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.mallocgc.func1()
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/malloc.go:1033 +0x3a
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.systemstack(0x6d4fdfc8)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/asm_386.s:399 +0x62
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.mstart()
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/proc.go:1146
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: goroutine 17 [running, locked to thread]:
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.systemstack_switch()
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/asm_386.s:360 fp=0x37d5b62c sp=0x37d5b628 pc=0x89c34f30
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.mallocgc(0x10000000, 0x8a7eecc0, 0x1, 0x89c1fcd5)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/malloc.go:1032 +0x6ff fp=0x37d5b680 sp=0x37d5b62c pc=0x89be686f
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: runtime.makeslice(0x8a7eecc0, 0x4000000, 0x4000000, 0x999c000)
2020-01-28 07:43:51.452 com.decred.dcrandroid.testnet E/Go: 	/usr/local/go/src/runtime/slice.go:49 +0x50 fp=0x37d5b694 sp=0x37d5b680 pc=0x89c1fd20
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/golang.org/x/crypto/scrypt.Key(0x2bdd65d0, 0x6, 0x6, 0x889f9f4, 0x20, 0x20, 0x40000, 0x8, 0x1, 0x20, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/golang.org/x/crypto/scrypt/scrypt.go:205 +0x120 fp=0x37d5b6f4 sp=0x37d5b694 pc=0x89e20c10
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl.(*SecretKey).deriveKey(0x889f9f0, 0x8a27981d, 0x12, 0x8d1bab0, 0x20, 0x20)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl/snacl.go:115 +0x6e fp=0x37d5b758 sp=0x37d5b6f4 pc=0x89e2156e
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl.NewSecretKey(0x8d1bab0, 0x40000, 0x8, 0x1, 0x2b653800, 0x9975e90, 0x89e3d415)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/internal/snacl/snacl.go:261 +0x1b8 fp=0x37d5b80c sp=0x37d5b758 pc=0x89e220b8
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.defaultNewSecretKey(0x8d1bab0, 0x8ad86450, 0x0, 0x0, 0x0)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/addressmanager.go:158 +0x3a fp=0x37d5b82c sp=0x37d5b80c pc=0x89e3d44a
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.createAddressManager(0x8a8d1020, 0x9977d80, 0x9975590, 0x20, 0x21, 0x2bdd65d0, 0x6, 0x6, 0x2bdd65d8, 0x1, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/addressmanager.go:2439 +0x850 fp=0x37d5ba38 sp=0x37d5b82c pc=0x89e4a9c0
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.Initialize.func1(0x8a8cdf60, 0x9977d60, 0x8a275c88, 0xe)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/initialize.go:34 +0x330 fp=0x37d5bab4 sp=0x37d5ba38 pc=0x89e82c50
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/walletdb.Update(0x8a8cabc0, 0x99779a0, 0x8a8caca0, 0x8d1b6e0, 0x37d5bb6c, 0x0, 0x0)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/walletdb/interface.go:249 +0x1bb fp=0x37d5bb44 sp=0x37d5bab4 pc=0x89e1c8ab
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb.Initialize(0x8a8cabc0, 0x99779a0, 0x8a8caca0, 0x8d1b6e0, 0x37c7c1c0, 0x9975590, 0x20, 0x21, 0x2bdd65d0, 0x6, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/udb/initialize.go:19 +0xbc fp=0x37d5bb9c sp=0x37d5bb44 pc=0x89e4e1fc
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3.Create(0x8a8cabc0, 0x99779a0, 0x8a8c8a10, 0x3bfe4520, 0x2bdd65d0, 0x6, 0x6, 0x2bdd65d8, 0x1, 0x8, ...)
2020-01-28 07:43:51.454 com.decred.dcrandroid.testnet E/Go: 	/Users/collins/go/src/github.com/raedahgroup/dcrlibwallet/vendor/github.com/decred/dcrwallet/wallet/v3/wallet.go:4466 +0x178 fp=0x37d5bbf0 sp=0x37d5bb9c pc=0x8a011128

As explained towards the end of the method, the key derivation operation consumes a bit of memory, which is not freed immediately the method returns. For that reason, code was added at the end of that function to force garbage collection and free up memory. This forced garbage collection doesn't seem to work on mobile devices, as unlocking wallets use up high memory, which does not get freed up, causing the devices to slow and eventually the app crashes at some attempts to unlock the wallet.

I get this error occasionally when tyring to pay a ticket fee. [PayVSPFee] err: failed to get votingKeyWIF for TsVVas5a18YE3CkWV1HFN81zwQ2Z2QFNbSP: wallet.DumpWIFPrivateKey:: wallet.LoadPrivateKey: no private key for default/1/123
This seems to be as a result of the wallet getting locked just before attempting to pay a fee.

Follow up to #57.

The wallet is considered to be open if wallet.internal != nil. wallet.internal is assigned whenever a wallet is created or opened using wallet.loader, which also sets the created/opened wallet to wallet.loader.LoadedWallet().

If there are scenarios where the wallet is unloaded but wallet.internal is not unset, the impression would be that there is an opened/loaded wallet, when there really isn't.

Even if such scenarios do not come up, it's quite redundant to store the same variable in 2 places when 1 will suffice.

Proposal is to remove the wallet.internal property and introduce a wallet.internal() alias method that return wallet.loader.LoadedWallet().

There should be a method on dcrlibwallet that returns the duration till next ticket price is determined.

While syncing i noticed fetched header count was fluctuating, for instance from 36000 to 34000 and then move to 38000 fetched headers. it appears to be caused by requesting headers from multiple peers and receiving the headers out of order, A solution to this may be only using the best known header so as not to go backwards.

Currently on dcrios, politeia sync completed successfully, but a proposal is stuck in the wrong category.

The update method needs to be confirmed

A function for fetching;

1. Vote results for all fetched proposals from #153
   A reference to the Api docs can be found here: https://github.com/decred/politeia/blob/master/politeiawww/api/www/v1/api.md#proposal-vote-status

2. Vote results for individual proposals using the censorship token
   A reference to the Api docs can be found here: https://github.com/decred/politeia/blob/master/politeiawww/api/www/v1/api.md#proposals-vote-status

Preferred response type would be a blocking response

A function for fetching all politeia proposals is needed.

This is the API link https://proposals.decred.org/api/v1/proposals/vetted

The preferred response for this would be a blocking response

There is a dcrlibwallet error while trying to create a wallet on the decred android mobile app, the attached screenshot describes the error.

I’m experiencing a bug when trying to create an additional wallet, the wallet stalls forever on canceling sync(log says "Address manager is shutting down"). It doesn’t stall if the sync is cancelled, from overview page and creating wallet works fine if sync is already canceled.

I renamed an account successfully but my transaction inputs & outputs are still showing the old account name. Account names should be loaded from the wallet when needed instead of saving it to the database to avoid this bug.

There are absolutely no tests currently and it's very disconcerting that a project of this size and purpose does not have a single test file.

Dcrlibwallet main package has a lot of code that needs to be cleaned up. The code structure in dcrlibwallet has been the same since the proof of concept code and now it’s a too rough and needs to be structured properly. The main module which currently consists of Multiwallet, Wallet, Politeia, TxIndex, VSPD will be divided into separate modules and packages.

Several reports have been received about transactions sent before/during first sync not being seen by the wallet. A wallet restore from seed has to be done to recover such transactions. A rescan does not solve it due to issue #71. So far we have been unable to recreate this bug by sending a transaction to a newly created wallet before the initial sync.

While dcrlibwallet multiwallet is designed to share one public passphrase across different wallets, it is currently possible for different wallets to use different public passphrase.

This is because a public passphrase is requested when creating new wallets or restoring existing wallets, but is not checked to match the public passphrase for previous existing wallets, making it possible for a user to specify a public passphrase different from that used by other wallets.

Dcrlibwallet has 3 transaction directions
SENT - 0
RECEIVED - 1
TRANSFERED - 2
These should be declared as constants so it can be accessible outside dcrlibwallet.

• ensure tx db data integrity by checking the hashes of first and last txs in the index db against the wallet db

  • this should be done after the wallet is opened but before syncing begins, so that if data mismatch is discovered, the index db can be deleted, recreated and re-populated when sync starts

• should register for tx notifications soon after sync completes rather than waiting for client apps to register a notification listener first. This is to ensure that new transactions information are received, processed and stored in the index db even if the client app has not registered a tx notification callback.

We need the client to send out notifications for certain events regarding politeia.

These events are listed below;

1. A notification should be broadcast when there is a new proposal
2. A notification should be broadcast when a vote for a proposal has started
3. A notification should be broadcast when a vote for a proposal has ended

The preferred response type for this function should be a callback response

An error with the message "json: cannot unmarshal string into Go struct field GetFeeAddressResponse.request of type dcrlibwallet.GetFeeAddressRequest". This seems to be as a result of the recent update to the VSPD source code.

As seen on the godoc there are a lot of stuff going on but not explained.

Proposal voting functionality will be added to godcr, dcrandroid and dcrios soon. The API to execute votes on proposals would need to be added to dcrlibwallet.

Transactions database should be cleared and reindex after every rescan. This was causing rescan to not be effective when trying to recover the correct state, and required users to reseed instead to see the correct balance and transaction history.

There was a report of this occurring on mobile. After the issue is confirmed on mobile, it needs to be resolved.

decred/dcrwallet#1622
decred/decrediton#2358

IsAddressValid function logs an error whenever an invalid address is passed. This should be removed since the error is not handled and a false boolean is returned for an invalid address.

The Transaction Notification listener registered for each wallet does not fire for mempool/unmined transactions but fires for confirmed transactions.

The implication is that the first time a wallet that has completed initial syncing learns of a new transaction is after the transaction is mined into a block.

UPDATE:
It seems new/unconfirmed tx notifications are broadcasted for transactions sent out of an account but not for received transactions.

There are several reports on dcrios about new transaction not being displayed on the app if the app was at the background when it is first recieved, The users had to rescan or restart the app before it displays the new transaction, This is because ios is disconnecting the app from internet.

Errors during rescan are currently only logged on dcrlibwallet, which could cause the app interface to stagnate - displaying the last rescan progress information and hoping to get more information that it will never get as rescan has errored.

While syncing if the sync is cancelled and the app is shutdown, on starting sync the ETA starts from 0 upwards which is not suppose to be.

This should help in detecting compatibility errors and fixing bugs that may come up after upgrading to a higher version of dcrlibwallet.

Calling GetTransactions takes up ~4 seconds to return ~800 transactions which is quite slow. Tracing down issue shows that all those seconds were spent receiving transactions from dcrwallet which makes dcrwallet the main cause of the issue.
A fix was implemented in #15 The implementation queries transactions from dcrwallet and saves it in storm db which is managed directly from dcrlibwallet. This makes it faster to query transactions and also allows us to query transactions based on attributes.

Steps to recreate this issue

1. Create a new wallet
2. Go to overview page and sync
3. Try switching to other pages or try interacting with the overview page while sync is ongoing

You'd notice that every other thing becomes unresponsive, you can't even cancel the sync

This Issue has been noticed on both dcrios and dcrandroid

We need a method for getting the available treasury balance. This seems to have been implemented on dcrd.
https://github.com/decred/decred-binaries/releases#dcrd-v160-rc1

Re-reading the sync progress notification code highlighted the following 2 issues:

• Stage 1 (headers fetching) gets done for all wallets before progressing to stage 2 whereas stage 2 (address discovery) does not get done for all wallets before proceeding to stage 3. After stage 1 is completed for all wallets, the first wallet goes through stages 2 and 3 before the next wallet goes through stages 2 and 3. The implication of this reporting is that the client app user interface shows progress reports for stage 2, then stage 3 (for 1 wallet), back to stage 2, then stage 3 again (for the next wallet) and so on until the last wallet.

• Overall sync ETA calculation currently appears to be done on a per-wallet basis rather than for all wallets. The implication of this would be a constantly changing estimated completion time on the UI.

godcr uses dcrlibwallet directly without needing gomobile. Readme should reflect this.

It would be nice if the README provided a bit more information than "A Decred wallet library written in golang for dcrwallet" and the build instructions. Specifically, what does this library provide on top of dcrwallet, whether it optimizes anything for mobile, etc.

Calling wallet.GetAccountBalance returns 0 for LockedByTickets value when using badger db. The bug can be recreated by restoring a seed that has locked funds and the LockedByTickets value will always be 0. I've tried rescanning and also purchasing a ticket from decrediton while the same seed wallet is online on dcrandroid and the LockedByTickets value is still 0.