Error when importing Fuelly formatted CSV about hammond HOT 20 OPEN

Once PR #88 is merged and a new docker image is built I believe this issue can be closed.

If possible, could either @aspapunktse or @ZHoob2004 test out the new image once it's built?

from hammond.

After some further debugging, I've identified the problem as user.Currency containing the fully expanded currency format "US dollars (USD" when accounting.UnformatNumber expects the format to be "USD".

I'm tempted to simply use a regex to extract the substring between the parentheses, but that feels inelegant

Regex form here

var rgx = regexp.MustCompile(`\((.*?)\)`)
totalCostStr := accounting.UnformatNumber(record[9], 3, rgx.FindStringSubmatch(user.Currency)[1])

Feels messy but it works (for USD, the one case I tested), and only runs when importing so very rarely.

from hammond.

The currency model actually contains a prop for the currency code. I'm going to see if I can get the code from the model using the currency value set for the user.

However, once you get past the locale error it seems the csv format may have changed.

@ZHoob2004 would you mind providing the first line on your csv export (headings)? I only have records in fuelly as recent as Oct 2021.

from hammond.

@ZHoob2004 does your csv contain expenses along with fillups, or just fillups?

from hammond.

I tried both with and without an expense line but wasn't getting past the locale issue.

I had the same locale problem when importing any of the examples from the akhilrex/hammond repo and those contain expenses

from hammond.

Good to know, once the #49 pull request is merged you should at least be able to import fillups.

from hammond.

@ZHoob2004 you should now be able to import using the generic import.

from hammond.

Thank you very much. I already managed to import mine with my hack a while ago but hopefully this will help others with non-fuelly data sets to import (since I had to reformat mine by hand)

from hammond.

Hi, just built a new docker from latest image but generic import option is not an option, only Drivvo.

Anyone able to share an Drivvo-exported CSV for reference since it is a paid option to export :( and it doesn´t like my Fuelly either.

from hammond.

Sorry, didn't catch the conversations here before now. I forgot to make a release after merging @alienp4nda's code. I created a new release now. It should be avaliable on docker in a couple of hours. Docker build failed, will fix tomorrow

from hammond.

I no longer have the data I used to test with, and it wasn't "true" fuelly data anyway, since I exported it from another app and reformatted it. There are 2 sample data sets in this issue that can be used to verify. These did not work when I last tested. akhilrex#11

from hammond.

No worries, and thanks for the data reference. I'll grab those and test those.

from hammond.

There still seems to be issues with with the generic imports when there is only a value when the tank is filled up but no value when the tank wasn't filled up. The generic import also doesn't take into account fillups and services mixed in the same file.

Adding the csv files from the issue by @ZHoob2004 to this issue.
Fuelly.Export.2.csv
Fuelly.Export.csv

from hammond.

I am running the latest docker build and experiencing the same issue.

Adding my Fuelly Export so you have a current file to work with: FuellyExport.csv

2024/01/20 19:19:07 /api/service/userService.go:29
[0.262ms] [rows:2] SELECT * FROM `user_vehicles` WHERE `user_vehicles`.`user_id` = "bd01996d-f215-41ef-be9d-d895d422e4ac"

2024/01/20 19:19:07 /api/service/userService.go:29
[0.380ms] [rows:1] SELECT * FROM `vehicles` WHERE `vehicles`.`id` IN ("7f10f7b1-b1ce-41f0-83c3-cf3225391d86","ce534264-b7c3-4a14-93db-621be1b69e93")

2024/01/20 19:19:07 /api/service/userService.go:29
[1.030ms] [rows:1] SELECT * FROM `users` WHERE `ID` = "bd01996d-f215-41ef-be9d-d895d422e4ac" ORDER BY `users`.`id` LIMIT 1

2024/01/20 19:19:07 [Recovery] 2024/01/20 - 19:19:07 panic recovered:
No Locale Info Found
/go/pkg/mod/github.com/leekchan/[email protected]/unformatnumber.go:21 (0x970e9c)
/api/service/fuellyImportService.go:67 (0x975025)
/api/service/importService.go:102 (0x977344)
/api/controllers/import.go:25 (0x97d3f8)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:165 (0x953aa1)
/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:99 (0x953a8c)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:165 (0x98ac63)
/api/main.go:69 (0x98ac4a)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:165 (0x953aa1)
/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:99 (0x953a8c)
[GIN] 2024/01/20 - 19:19:07 | 500 |    4.417313ms |       10.2.1.26 | POST     "/api/import/fuelly"
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:165 (0x952c06)
/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:241 (0x952be9)
/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:165 (0x951cbd)
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:489 (0x95196b)
/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:445 (0x9514a4)
/usr/local/go/src/net/http/server.go:2936 (0x6bdb75)
/usr/local/go/src/net/http/server.go:1995 (0x6b9091)
/usr/local/go/src/runtime/asm_amd64.s:1598 (0x46f120)

from hammond.

The "No Locale Info Found" error suggests that you're running an older build. Please verify you're running v0.0.24.

from hammond.

I tried running both alfhou/hammond:latest(docker hub) and ghcr.io/alfhou/hammond:latest(github) images. They both are showing the same Digest (sha256:c85dc0293aa10be15280277bade9cce808be572d0d4c5eac62acd87dec30479d).

Both images show "2022.07.06" in the More Info section of the Settings page. Should that be showing "v0.0.24" instead?

docker pull alfhou/hammond:latest
latest: Pulling from alfhou/hammond
Digest: sha256:c85dc0293aa10be15280277bade9cce808be572d0d4c5eac62acd87dec30479d
Status: Image is up to date for alfhou/hammond:latest
docker.io/alfhou/hammond:latest

docker pull ghcr.io/alfhou/hammond:latest
latest: Pulling from alfhou/hammond
Digest: sha256:c85dc0293aa10be15280277bade9cce808be572d0d4c5eac62acd87dec30479d
Status: Downloaded newer image for ghcr.io/alfhou/hammond:latest
ghcr.io/alfhou/hammond:latest

from hammond.

Verified this is an issue, this will take me a bit to troubleshoot.

from hammond.

There are two related issues causing this.

The currency is saved as "Currency Name (CODE)". The settings page does this as does the initialize page on a brand new instance. The backend is expecting only the currency code. I have a fix but this will require everyone who has an existing instance to go to the settings page and select their currency again then save the settings.

from hammond.

It's up to you which way to go, but it looks like the issue is only affecting me at the moment. If you could share a workaround, I could import my data and not inconvenience everybody else.

There are two related issues causing this.

The currency is saved as "Currency Name (CODE)". The settings page does this as does the initialize page on a brand new instance. The backend is expecting only the currency code. I have a fix but this will require everyone who has an existing instance to go to the settings page and select their currency again then save the settings.

from hammond.

It will not just affect you, currently any backend code that needs to use the currency and is expecting the currency code will be affected. I'll just need to figure out a way to notify users.

from hammond.