hahnec / plenopticam Goto Github PK

The type is .jpg, but it can't work

On raw images taken with a modified Nikon camera, in comp_bayer(), the line a0 = (t0 << 4) + ((t1 & 0xF0) >> 4) gives

(ValueError('operands could not be broadcast together with shapes (13408599,) (13408598,) '),)

i don't really get what bay_img is but if it's the bayer image, you could do self._bay_img = rawpy.imread(self._lfp_path).raw_image.astype('float') innstead of calling self.comp_bayer(). (It uses the rawpy library)

Hi

I got sub-apertures data, named by j_i.png

I would like to know what's the meaning of (j, i)

Can I explain 'j' as a vertical index of a camera position and 'i' as a horizontal index of a camera position?

About the use of the Lytro Illum camera, I have a little question to ask you. Often, when I take a picture and look at it, there is an error mark (a cross), it cannot be displayed properly. What is the reason for this and how to avoid it? Looking forward to your reply!

hello, I have a question I'd like to discuss with you. In your code, we can see that some parameters used in the disp2pts function are default parameters, but in fact for Illum, these parameters are not applicable, we need calibrate the LF camera, how to get the correct parameters according to the calibration results?

I was trying to run the macos version, i downloaded the plenopticam_0.9.3.dmg.

the dmg opens fine, but trying to run the enclosed plenopticam.app gives an error saying it is damaged.

ive tried this on an i5 and m2 cpu.

any ideas?

Hello!

I've recently been getting involved in researching application using plenoptic cameras and depth estimation, and I came across this Python library! I am trying to run the OpEx dataset, using the calibration and image .BMP file located at the root of the dataset directory (f197Inf9pxFinalShift12.7cmf22.bmp and f197Inf9pxFinalShift12.7cm.bmp), but I am getting an error about a NoneType for self._wht_img.

(heka) kelvinchan@Quadro:~/Desktop/OpEx$ plenopticam -g

Plenopticam 0.2.0 by Christopher Hahne


 Starting ...

 Provided file /home/kelvinchan/Desktop/OpEx/f197Inf9pxFinalShift12.7cm/f197Inf9pxFinalShift12.7cm.json does not exist

 White image file not present
Exception in Tkinter callback
Traceback (most recent call last):
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/tkinter/__init__.py", line 1702, in __call__
    return self.func(*args)
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/tkinter/__init__.py", line 746, in callit
    func(*args)
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 147, in poll
    self.cur_thread.join()
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 341, in join
    raise misc.errors.PlenopticamError(self.exc, cfg=self.cfg, sta=self.sta)
plenopticam.misc.errors.PlenopticamError: Traceback (most recent call last):
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 333, in run
    self.ret = self._target(*self._args, **self._kwargs)
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 256, in cal
    cal_obj.main()
  File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/lfp_calibrator/top_level.py", line 50, in main
    self._wht_img = rgb2gray(self._wht_img) if len(self._wht_img.shape) is 3 else self._wht_img
AttributeError: 'NoneType' object has no attribute 'shape'

It also complains about the missing JSON for the calibration image, but that does not exist in this dataset. If this issue is due to the missing JSON, what are the values that Plenopticam expect in the calibration JSON?

Hi there,

I've just shared a writeup and a piece of code that enables direct access to all the features of the original Lytro camera.
You can find the writeup here https://github.com/ea/lytro_unlock and the code here https://github.com/ea/lytroctrl

Sharing in case somebody is still playing with these and has a cool idea that would benefit from easier control of the camera.

Cheers

Hello, I am a beginner at this and trying to go through your work as much precisely as possible. I wanted to recreate all your steps with one of your light field image and using your tar file. If i get successful I want to take images with our own plenoptic camera and recreate all the steps to get a clearer image.

I am getting error and can't really understand. Can you please help? adding the error message with this.

When I ran sudo python3 setup.py install,

No module named 'color_space_converter'

came out in the /plenopticam/misc/data_proc.py line 26.

I searched 'color_space_converter' module on the Internet, but I couldn't find it.

How can I solve this problem?

Hello,

The results are saved in a folder located at the same directory as the light field file.
I want to know how I can save them on a different directory.

Thank you

Hello, I want to ask some help. My original data is from Lytro Illum. The setting part is shown in the figure. Micro image patch size: 7; refocusing range: -2, 4. But After the progress bar of Light filed aligment reaches 76%, it stoped. I am confused . Can you give me some suggestions?

Hi,

I just got a Lytro Illum and I'm starting doing some test using plenopticam.
For what I understand the lytro illum best resolution output is around 4Mpx after processing but the output of plenopticam is images of 622x432 pixels.

Is there a way to have a better resolution out of plenopticam ?
I'm guessing the setting "Micro image patch" size might be what I'm looking for but the description is a bit short for me understand fully.

Anyway, thanks to make this tool available. My final goal is to make a 3d scanner based on photos from the illum and plenopticam is a real good starting point for the pre-processing.

Hi, I keep on getting the White image file not present error message when I use the raw.lfp and tar calibration data obtained from my Lytro Illum camera. But when I use the data downloaded from the INRIA Dataset, it works completely fine. I am using plenopticam v0.8.2. I have a macOS 12.3 Monterey with M1 chip.

Here is the Google Drive link for my two files.

Full error message:

 Loading data

 Potential data loss as directory already exists.
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/top_level.py", line 64, in main
    self.decode_lytro_file()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/top_level.py", line 106, in decode_lytro_file
    obj.main()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/lfp_decoder.py", line 71, in main
    self.decode_lfc()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/lfp_decoder.py", line 104, in decode_lfc
    sec_idx = self.get_idx(sections, int(self._shape[0] * self._shape[1] * self.cfg.lfpimg['bit'] / 8))[0]
KeyError: 'bit'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 366, in run
    self.ret = self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 277, in load_lfp
    lfp_obj.main()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/top_level.py", line 73, in main
    raise LfpTypeError(e, cfg=self.cfg)
plenopticam.misc.errors.LfpTypeError: 'bit'
Exception in thread Thread-5:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 366, in run
    self.ret = self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 261, in auto_find
    obj.main()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/cali_finder.py", line 105, in main
    self._search_tar_file(self._path)
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/cali_finder.py", line 265, in _search_tar_file
    self._match_georef(json_dict)
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/cali_finder.py", line 166, in _match_georef
    for item in json_dict[key1]:
KeyError: 'frame'

 White image file not present
Exception in thread Thread-6:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 366, in run
    self.ret = self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 252, in cal
    cal_obj.main()
  File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/top_level.py", line 58, in main
    if len(self._wht_img.shape) == 2 and 'bay' in self.cfg.lfpimg:
AttributeError: 'NoneType' object has no attribute 'shape'

hi，Hahne! I am very interested in your project. I run plenopticam under win, and when the program executes to light field alignment 47%, it doesn't move. Can I save sub-aperture images and how to save?

Hi, thanks for this project so I can try out real refocusing with light-field photographs.

It is painfully slow running on Windows and the CPU is certainly not fully used, is memory accessing the bottleneck in this case?

And the result is certainly much better comparing to Lytro Desktop in terms of accuracy. This makes my wonder the reason why Lytro Desktop didn't choose this way to process images? It seems to generate images by fusing preprocessed images with different depths.

Thanks again!

Currently SciPy version 1.4.1 is required (introduced in this commit: 0d67816) which is a bit undesirable

It would be great if this version requirement was removed - release 1.6.0 might have solved the issue.
In case the issue is still there.. is it an issue which could be fixed at this repository?
If you can provide some details about it @hahnec I might try to fix it. Cheers.

Hey man! Awesome updates!!!!!

I'm very excited to view the 3D point cloud data but I'm having a hard time finding a free (or paid) Windows application that will do it.

Do you have any ideas?

Does anyone know if this software works with the original Lytro Camera? The cuboid one.

Hi again!

While I'm working with the v0.3.0 test version, I notice that I get a circle P in the export images and .gifs when I am trying to process a set of lightfield images from a custom microlens camera, which is not the case when I was processing the provided example dataset.

What would this watermark mean in particular? Does it mean a bad lightfield image, an improper calibration process, failure to decompose, or something else? On other lightfield images tested, I also sometimes see green or black background images in the exported viewpoint stack.

Hi again!

Hope you are doing well amidst all the recent happenings with the coronavirus outbreaks! I'm creating another issue here to follow up on the discussion last time on implementing focused plenoptics setup 😄

Being stuck at home these days, I finally got around to looking at the focused plenoptics setup on the papers you suggested! If I'm understanding it properly, spatial-angular resolution in the focused setup is determined by the ba system instead from , where b is the distance between the sensor and the MLA, and a is the imaging distance of the microlens (aligned to the image formed by the main lens). Spatial resolution for the subaperture image would be the sensor resolution multiplied by (b/a), and angular resolution would be (a/b).

I found another software by Max, RayTracer, that lets me simulate MLA images in the focused setup, at least that is what's claimed here. For his software, f-number matching is locked at a 95% ratio between the main lens and MLA.

The parameters for my mockup camera that I used to render the resulting synthetic images are as follows:

Sensor Resolution: 4096x3072 px
Pixel Size: 3.2 um
Main Lens Diameter: 7 mm
Main Lens F-Number: 16
Main Lens Focal Length: 112 mm
MLA Lens Diameter: 66 um        (from my actual MLA film)
MLA F-Number: 15.15             (from my actual MLA film)
MLA Focal Length: 1 mm          (from my actual MLA film)
Image Distance: 137.7 mm
Focus Distance: 600 mm

# "staircase" is placed ~3.5 m in front of camera

One thing I'm curious about, why might we want to do f-number matching for the MLA and main lens in a focused setup (as suggested by Lytro designs)? What if another f-number for the main lens is chosen?

But besides that, the real topic for this issue would be on how to go about adding a new feature on Plenopticam to decode this focused plenoptics image.

Is it possible to control (or estimate) the depth of field of the output images? e.g. is it possible to request an image that would be equivalent of a lens operating at f/8? or is it possible to estimate the equivalent f-stop of an output image?

Plenopticam fails at Debayering stage on Ubuntu.
Already tried:
Ubuntu 18.04 with Python 3.8.10;
Ubuntu 20.04 with Pyhton 3.8.10;
Ubuntu 20.04 with Python 3.9;
Ubuntu 22.04 with Python 3.10.

Thank you

It seams this is a reacquiring problem. I get this error with version 0.9.3.

left image is produce by your work, the right one is produce by another. I found the left image is more darker than another. Is it normal? or has errors with my setting?

Hi, I'm trying to setup in order to test Lytro content on the LookingGlass Portrait.
I tried python3 -m pip install plenopticam but ended up in dependency hell. After a bit of search I decided to use pyenv to rollback from Python 3.9 to 3.8 as apparently scipy had problem with it (no package yet). That ended up solving this part but then few more dependencies were still missing, namely sciki-image, which was fine, but also tk-tkinter which I didn't manage to solve :

  File "/home/fabien/.pyenv/versions/3.8.12/lib/python3.8/tkinter/__init__.py", line 36, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk

As it gets a bit hairy trying to mix pyenv and system wide setups e.g sudo apt install python3-tk tk-dev I'm wondering if a dockerfile wouldn't simplify. What I did for now was :

docker run -it --rm  scipy/scipy-dev
python3.8 -m pip install scikit-image
python3.8 -m pip install plenopticam
plenopticam -h

which worked.

Hello， Can I use this code for my own LFM？

I'm always getting this same error message, no matter if I select the Illum tar calibration file or the folder containing the calibration files. The correct white image files are all in there. I'm using an lfr image as my light field image. This is with version 0.8.0 and the Illum on the latest firmware version.

Hello, will you work in others plenoptic camera in a near future ?

I would like to test 'Plenopticam', but I don't have calibration data.

While searching on the Internet, I can only find lfp raw images without calibration data.

Would you let me know where can I get lfp raw images including calibration data?

It doesn't matter if the number of raw images is small.

Hello,

I experienced this issue whenlaunching 0.5.3 for the first time on Windows, after installing.

I also have the “white image file not present” error. Running v0.91b in windows 10, load “raw.lfp” for image and “caldata-<the camera’s serial number>.tar” for calibration source and get white image not present error. My camera cannot be updated to version 2.0 firmware and is still on version 1. I am able to view images and use all features in the lytro desktop software and I have used matlab lightfield toolbox to view and alter the images after importing the calibration data into the toolbox. I suspect the problem is related to the firmware version of my camera, as experienced by users in issues 22 and 25. My camera serial number starts with number rather than letter.

sir，I want to measuring the dimensions of microscopic samples,like this http://lightfield-forum.com/wordpress/wp-content/uploads/2018/12/lightfield-feature-depth-mapping-distance-measurement-2.jpg,could you please tell me how to convert depth map to absolute distance? thank you very much

I still have a question, that is, the generated refocused images are generally darker. Is this normal? or have problems with my settings?

Originally posted by @hongyann in #2 (comment)

Hi again!

While waiting for the other issue to be fixed, I have been looking over the other features within this library - are there any plans to add features for generating depth/disparity maps from a calibrated lightfield image? As I mentioned, I'm currently looking into a custom lightfield camera and trying to extract depth information from it.

If it will not be part of the scope of this library, how might I continue onward after getting a calibrated lightfield? I am not all too familiar with the typical approach to generating disparity from light fields, so any guidance on how to do so would be much appreciated!

On another note, another plenoptic library (plenpy) that I've found has this feature implemented. The only issue with their library is that their generic lightfield camera implementation is incomplete... Would it be possible to export the calibrated lightfield data from this library and reload the lightfield on plenpy?