Clip error about licsbas HOT 16 CLOSED

from licsbas.

And also, if I don't use the clip step, it will end with this step:
Process 1345/2690th line (1/2th patch)...
Reading 66 ifg's unw data...
1327736/4188330 points removed due to not enough ifg data...

Identifing gaps and counting n_gap...
Counting n_ifg_noloop...

Small Baseline inversion by LS...

Solving 1341331/2860594th points with full unw at a time...
Next, solve 1519263 points including nan point-by-point...
Running 1519000/1519263th point...
Elapsed time for 1th patch: 00h 06m 54s

Process 2690/2690th line (2/2th patch)...
Reading 66 ifg's unw data...
1633923/4188330 points removed due to not enough ifg data...

Identifing gaps and counting n_gap...
Counting n_ifg_noloop...
(base) weiyi@weiyi-virtual-machine:~/Desktop/LiCSBAS/114A_05790_131313$

from licsbas.

Can you upload full log messages for the first issue?

The second issue might be caused by memory over. This step could use larger amount of memory than expected. Please retry the step 1-3 with, e.g., p14_mem_size="2000" in batch_LiCSBAS.sh.

from licsbas.

Have you solved this issue? @tanmeimei

from licsbas.

I haven't solved this issue and here is the log file
202007081414batch_LiCSBAS_01_16.log

from licsbas.

I investigated the files and realised that the file size of [ENU].tif and hgt.tif is inconsistent with the unw.tif. This problem is in the LiCSAR processing and the files need to be recreated. I have asked the person in charge. I hope this problem will be fixed soon.

from licsbas.

hi tanmeimei,

thank you for reporting this issue.
there was an error in the frame files. we are working on fixing it and should generate actually much more interferograms soon!
hope it is ok if you can wait for the data a week or two (please let us know if this is urgent)

from licsbas.

Thanks, this is not very urgent.

from licsbas.

It took a long time but now the data in 005D_04801_131313 are available. Please try again.

from licsbas.

I meet the same problem in area 063D_04886_161920. An error occurred (cannot reshape array of size 2337741 into shape***) in step05 or step13. It seems that the file size of [ENU].tif and hgt.tif is inconsistent with the unw.tif.

from licsbas.

thank you for reporting this issue with the LiCSAR data, frame 063D_04886_161920. I see there were 14 ifg pairs of the frame with this issue. We fixed this by removing those pairs and we will investigate the source of the error. Please rerun your processing chain, or just delete following directories from the GEOC folder: 20200505_20200517 20200529_20200622 20200622_20200716 20200704_20200728 20200728_20200809 20200728_20200902 20200809_20200821 20200809_20200902 20200809_20200914 20200821_20200902 20200821_20200914 20200902_20200914 20200902_20210217 20200914_20210217 út 25. 7. 2023 v 4:53 odesílatel Feng-Nian Chang @.> napsal:
…
I meet the same problem in area 063D_04886_161920. An error occurred (cannot reshape array of size 2337741 into shape) in step05 or step13. It seems that the file size of [ENU].tif and hgt.tif is inconsistent with the unw.tif. — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYYKQ65BQN3HHOAV4Y5IDXR47NFANCNFSM4ONPU24Q . You are receiving this because you commented.Message ID: @.***>

Thank you for your prompt reply! However, the error still occurred after deleting the problematic directory. In the step5 (Clip) it says:

And also, if I don't use the clip step, it will end with step13 (NSBAS inversion):

The problem has not been solved. In fact, I have removed these unwrapped data with inconsistent sizes in the last processing. So it seems that the problem is not because of this, I guess it may have something to do with [ENU].tif and hgt.tif?

Best regards!
Fengnian Chang

from licsbas.

from licsbas.

hi, indeed, the problem remained in the latest mli.tif files on LiCSAR side. fixed now. M. st 26. 7. 2023 v 3:14 odesílatel Feng-Nian Chang @.> napsal:
…
thank you for reporting this issue with the LiCSAR data, frame 063D_04886_161920. I see there were 14 ifg pairs of the frame with this issue. We fixed this by removing those pairs and we will investigate the source of the error. Please rerun your processing chain, or just delete following directories from the GEOC folder: 20200505_20200517 20200529_20200622 20200622_20200716 20200704_20200728 20200728_20200809 20200728_20200902 20200809_20200821 20200809_20200902 20200809_20200914 20200821_20200902 20200821_20200914 20200902_20200914 20200902_20210217 20200914_20210217 út 25. 7. 2023 v 4:53 odesílatel Feng-Nian Chang @. > napsal: … <#m_-1295850161274870515_> I meet the same problem in area 063D_04886_161920. An error occurred (cannot reshape array of size 2337741 into shape) in step05 or step13. It seems that the file size of [ENU].tif and hgt.tif is inconsistent with the unw.tif. — Reply to this email directly, view it on GitHub <#25 (comment) <#25 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYYKQ65BQN3HHOAV4Y5IDXR47NFANCNFSM4ONPU24Q . You are receiving this because you commented.Message ID: @.> Thank you for your prompt reply! However, the error still occurred after deleting the problematic directory. In the step5 (Clip) it says: [image: 1] https://user-images.githubusercontent.com/62586311/256085134-330f92d4-869e-4fd3-b36d-abcf7cef0074.png And also, if I don't use the clip step, it will end with step13 (NSBAS inversion): [image: 2] https://user-images.githubusercontent.com/62586311/256085408-20cabba2-b663-43e7-b235-5cc2df40baa5.png The problem has not been solved. In fact, I have removed these unwrapped data with inconsistent sizes in the last processing. So it seems that the problem is not because of this, I guess it may have something to do with [ENU].tif and hgt.tif? Best regards! Fengnian Chang — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYYKSUUDLXBM6I2XMTTE3XSB4P5ANCNFSM4ONPU24Q . You are receiving this because you commented.Message ID: @.***>

Thank you and your team for your efforts to provide such a good InSAR product, looking forward to your early reply.

from licsbas.

hi, indeed, the problem remained in the latest mli.tif files on LiCSAR side. fixed now. M. st 26. 7. 2023 v 3:14 odesílatel Feng-Nian Chang @.> napsal:
…
thank you for reporting this issue with the LiCSAR data, frame 063D_04886_161920. I see there were 14 ifg pairs of the frame with this issue. We fixed this by removing those pairs and we will investigate the source of the error. Please rerun your processing chain, or just delete following directories from the GEOC folder: 20200505_20200517 20200529_20200622 20200622_20200716 20200704_20200728 20200728_20200809 20200728_20200902 20200809_20200821 20200809_20200902 20200809_20200914 20200821_20200902 20200821_20200914 20200902_20200914 20200902_20210217 20200914_20210217 út 25. 7. 2023 v 4:53 odesílatel Feng-Nian Chang @. > napsal: … <#m_-1295850161274870515_> I meet the same problem in area 063D_04886_161920. An error occurred (cannot reshape array of size 2337741 into shape) in step05 or step13. It seems that the file size of [ENU].tif and hgt.tif is inconsistent with the unw.tif. — Reply to this email directly, view it on GitHub <#25 (comment) <#25 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYYKQ65BQN3HHOAV4Y5IDXR47NFANCNFSM4ONPU24Q . You are receiving this because you commented.Message ID: @.> Thank you for your prompt reply! However, the error still occurred after deleting the problematic directory. In the step5 (Clip) it says: [image: 1] https://user-images.githubusercontent.com/62586311/256085134-330f92d4-869e-4fd3-b36d-abcf7cef0074.png And also, if I don't use the clip step, it will end with step13 (NSBAS inversion): [image: 2] https://user-images.githubusercontent.com/62586311/256085408-20cabba2-b663-43e7-b235-5cc2df40baa5.png The problem has not been solved. In fact, I have removed these unwrapped data with inconsistent sizes in the last processing. So it seems that the problem is not because of this, I guess it may have something to do with [ENU].tif and hgt.tif? Best regards! Fengnian Chang — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADYYKSUUDLXBM6I2XMTTE3XSB4P5ANCNFSM4ONPU24Q . You are receiving this because you commented.Message ID: @.***>

I had the same problem again in frame 056A_04947_282019 and 136D_04941_172131. Maybe there is a problem with the [ENU].tif, hgt.tif, or mli.tif?
@yumorishita @espiritocz
202309170824batch_LiCSBAS_01_16.log
202309180358batch_LiCSBAS_02_16.log

BTW, frame 136D_04941_172131 does not exist in the LiCSAR visualization map (https://comet.nerc.ac.uk/COMET-LiCS-portal/).

from licsbas.

from licsbas.

@espiritocz
Thank you for your reply and suggestion! This error may not be caused by the coherence threshold.

I mask in step 04 to preserve more ifgs in step 12. otherwise many ifgs would be discarded due to large loop closure errors.
The COMET version you recommend seems to improve this by nullification of pixels rather than dropping ifgs. This is a big improvement and will be worth a try.
However, I ran into problems installing the COMET version in the same way. Is there any installation instructions or manuals?

Additionally, frame 056A_04947_282019 has the same error in step 03.
202309180358batch_LiCSBAS_02_16.log

Best regards!
Fengnian Chang

from licsbas.