baruchiro / israeli-bank-scrapers-desktop Goto Github PK

After PR #66 will merge, we will have a strange flow to use the SCRAPERS from the israeli-bank-scrapers-core in the renderer process.

From the comment:

I don't understand why the SCRAPERS from israeli-bank-scrapers-core needs to imported in scrapers.service, then imported by background.js, who is expose them through the ipc, and then this Importers.vue call them to add them to the store, then uses them by state and getters.

@Arielgordon123 says that there is a conflict with ws.

We need to solve this or open a bug in israeli-bank-scrapers

This issue is for discussion of the entire password and key security process.

• Password shouldn't shown as clear text
• Generate SALT at the build or install
• Use keytar for saving passwords
• Filter passwords from persistent data
• Don't store the encryption key and passwords in the same place

So that the user can see which row in the table came from which Importer.

Maybe colors?

Related to #53

How the object interpolations, Vuex state management and app state saving should work together?

As @esakal said, We use yarn as the package manager, but in the README I wrote npm commands.

To solve this issue, need to clone a fresh clone, and see which yarn commands we need to run until we run the dev, test and build:repease, and write these commands in the README.

The commands can also be taken from the CI (yml) files.

Each Importer identified by the Importer ID, that unique to the provider and not to Importer.

Today, we save the SALT with keytar, then using this SALT to encrypt the passwords, and saving the passwords alongside the SALT.

This means that there is almost no meaning for encryption, because if they can access the encrypted passwords, they can also access the encryption key.

Currently, the passwords saved encrypted in the filesystem, but the algorithm and the SALT are in the source code.

We have to generate the SALT at build time, or even better- at the install time (but what will append at upgrade time?)

Until we will Replace the vuex-electron plugin, we need to filter the sensitive mutations from the persistent file.

See black-list and white-list in vuex-electron.

I think the way to implement this is to add a button called "Validate", for example, to execute the Importer action before we adding the Importer, and do nothing with the results, to validate we inserting the right values.

Allow editing the Importer instead of deleting and readding.

currently there is no indication at first time when downloading chrome
this step could take some time depending on the user network

electron-builder throws this error:

Error: GitHub Personal Access Token ("***") contains invalid characters, please check env "GH_TOKEN"

There are workarounds here, something with Regex replaces.

The real solution is to upgrade electron-builder to ^22, but it requires to upgrade the vue-cli-plugin-electron-builder, as described here.

If we upgrading to the v2.0.0-beta.0, it can solve also the #68 , because it allows us to upgrade the electron version and inject the spectron.

We need a button to clear the store data.

When the user will click on this button:

• Popup- are you sure?
• Show how many transactions will delete
• Which more data will should delete?
• Delete the data.

It seems that every node process can retrieve passwords stored by keytar by our process.

This is a limitation that is described in the keytar bug, and it seems to be a security weakness that software generally does not want to take responsibility for (Chrome, for example).

I think we should keep all passwords secure with a user password.
For example, use the user password as an encryption key.

Need to create a Hebrew document with FAQ and Troubleshooting,

I saw that there is a package called keytar and is designed to store passwords with the Electron app.

The Electron-Vue boilerplate starts with lower versions of Vue and Electron.

• Electron is now 2.0.4, should be ^5.0.0.
• Vue is now 2.5.16, should be ^2.6.0.
• nedb should be removed.

Consider adding more tests before you upgrade the dependencies (#4).

When we publish a new release?

Every push to master?
Manually?

Anyway, I need to implement a basic pipeline for beta work.

Implement an Auto Update mechanism to notify the user of a new version and allow it to update directly.

Do not update the app without user permission!

• Work only with getters, don't use mapState directly.

Now we uploading the transactions to the Google Spreadsheet by

transactionsArray.map((transaction) => mapKeys(transaction, (_value, key) => key.toLowerCase()))
  .forEach(async (transaction) => {
    await addRow(worksheet, transaction);
  });

This can throw this error:

Uncaught (in promise) Error: HTTP error 429 (Too Many Requests)

So we need to replace the GoogleSpreadsheet.addRow with GoogleSpreadsheet.bulkUpdateCells.

from @Arielgordon123 :

See this bug. What is the gnome-keyring?

• Solve the bug.
• Write a test for saving a password with keytar in CI only.

Today the Build/Release pipeline triggered by any push to master.

Use path to filter and trigger for only relevant files.

The relevant files are:

• src
• .electron-vue

And maybe babel and webpack files?

Debug is a word with a programming context. For the users, we need to replace the Checkbox in the importers with Show Browser or something.

When I click "import" with "show browser" on I see it scrapes data from 10/01/19 - 10/01/2020.
Once the import is done I see all the data from the sheet are in the range 31/12/20 - 02/01/2020.
of course the dates are wrong since today is 10/01/2020.
I looked at the content of the cells and reached the conclusion it shows all the 2019 dates as 2020.

the library libsecret need to be install manually on linux

• Should I move to another test framework?
• Create first real test that validate a component with faked Vuex and faked child components.

I don't know enough how to design a node-electron-Vue app to work with interfaces. The next subjects should be redesigned to be replaceable:

• Importers- How can I enable the option to easily add more importers to the App?
• App DB- now I use nedb in the jsonFileDB.js file. This is an implementation of a future interface for saving the App state.

Now, when clicking on the link in the SpreadsheetExporter, it opening a new Electron window to show the web-page.

According to this it should open a browser window, and probably you need to update the dependencies first.

When we adding an Importer (Scraper), we need to meet the following requirements:

• When the user clicks on Add, the Importer should appear on the screen. Currently, the Importer added under the "Add Scraper" sub-menu, so when the user clicks "Add", they see no effect (except that the fields are being empty).
• Consider whether to collapse the current "Add Scraper", or the whole sub-menu, after adding an Importer.
• Paging in the main table.

The vuex-electron package is not under maintaining anymore.

In this app, we need more attention about persisting the vuex state, because we have to keep the sensitive data (passwords, keys) out of the persistent file (unless we keeping the whole persistent safe).

So this issue requires looking for maintained, customizable and secured Vuex plugin, or developing our own plugin.

• Add my email: [email protected] to the UI, with "Send Feedback" or something (link to repo...) (#113)
• Add options to send the logs to somewhere (but let the user see them before he sends them).

Add one button to import with all registered importers.

We need to think about how to implement this. Don't start to work without discussing it.

At the beginning of the README, we should clearly state the division of responsibilities between us and the Scrapers library.

As long as we don't do this, we will have to answer a lot of questions related to the Scrapers library (and even after we will do it ;-))

When we adding an Importer, we saving his details with keytar.

The password manager will be contaminated if we do not delete the passwords when we remove Importer.

In README, we need to add an explanation how to init a developer workspace:

• Install node (version?)
• Install Yarn
• Project structure
• Trubleshooting

The snap package didn't start with this error:

/snap/israeli-bank-scrapers-desktop/3/israeli-bank-scrapers-desktop: error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory

I need to add a logger for getting more details when something don't work as expected, but without loading the console.

In PR #66 we turned off the e2e tests on Windows, because they didn't end.

See this comment.

Our users speaking Hebrew, so I think we need to support:

• RTL
• Translate to Hebrew
• All strings in one location

When I try to Export to Google Spreadsheet I get this error (it's shown when I hover over the small red X to the right of "Export to Google Spreadsheet" ):
Cannot read property '0' of null

and the log shows:

┏ Electron -------------------

14:34:03.076 › TypeError {}

┗ ----------------------------

Currently, we ask the user to share his spreadsheet with the App Google Service account,

Maybe getting permissions to the user GDrive is more simple (for the user).

Please add links to how to implement this.