hahnec / plenopticam Goto Github PK
View Code? Open in Web Editor NEWLight-field imaging application for plenoptic cameras
Home Page: http://www.plenoptic.info
License: GNU General Public License v3.0
Light-field imaging application for plenoptic cameras
Home Page: http://www.plenoptic.info
License: GNU General Public License v3.0
Hey man! Awesome updates!!!!!
I'm very excited to view the 3D point cloud data but I'm having a hard time finding a free (or paid) Windows application that will do it.
Do you have any ideas?
Hi, I keep on getting the White image file not present
error message when I use the raw.lfp and tar calibration data obtained from my Lytro Illum camera. But when I use the data downloaded from the INRIA Dataset, it works completely fine. I am using plenopticam v0.8.2. I have a macOS 12.3 Monterey with M1 chip.
Here is the Google Drive link for my two files.
Full error message:
Loading data
Potential data loss as directory already exists.
Exception in thread Thread-4:
Traceback (most recent call last):
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/top_level.py", line 64, in main
self.decode_lytro_file()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/top_level.py", line 106, in decode_lytro_file
obj.main()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/lfp_decoder.py", line 71, in main
self.decode_lfc()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/lfp_decoder.py", line 104, in decode_lfc
sec_idx = self.get_idx(sections, int(self._shape[0] * self._shape[1] * self.cfg.lfpimg['bit'] / 8))[0]
KeyError: 'bit'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 366, in run
self.ret = self._target(*self._args, **self._kwargs)
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 277, in load_lfp
lfp_obj.main()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_reader/top_level.py", line 73, in main
raise LfpTypeError(e, cfg=self.cfg)
plenopticam.misc.errors.LfpTypeError: 'bit'
Exception in thread Thread-5:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 366, in run
self.ret = self._target(*self._args, **self._kwargs)
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 261, in auto_find
obj.main()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/cali_finder.py", line 105, in main
self._search_tar_file(self._path)
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/cali_finder.py", line 265, in _search_tar_file
self._match_georef(json_dict)
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/cali_finder.py", line 166, in _match_georef
for item in json_dict[key1]:
KeyError: 'frame'
White image file not present
Exception in thread Thread-6:
Traceback (most recent call last):
File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 366, in run
self.ret = self._target(*self._args, **self._kwargs)
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/gui/widget_ctrl.py", line 252, in cal
cal_obj.main()
File "/opt/homebrew/lib/python3.9/site-packages/plenopticam/lfp_calibrator/top_level.py", line 58, in main
if len(self._wht_img.shape) == 2 and 'bay' in self.cfg.lfpimg:
AttributeError: 'NoneType' object has no attribute 'shape'
Hello, will you work in others plenoptic camera in a near future ?
I also have the “white image file not present” error. Running v0.91b in windows 10, load “raw.lfp” for image and “caldata-<the camera’s serial number>.tar” for calibration source and get white image not present error. My camera cannot be updated to version 2.0 firmware and is still on version 1. I am able to view images and use all features in the lytro desktop software and I have used matlab lightfield toolbox to view and alter the images after importing the calibration data into the toolbox. I suspect the problem is related to the firmware version of my camera, as experienced by users in issues 22 and 25. My camera serial number starts with number rather than letter.
When I ran sudo python3 setup.py install
,
No module named 'color_space_converter'
came out in the /plenopticam/misc/data_proc.py line 26.
I searched 'color_space_converter' module on the Internet, but I couldn't find it.
How can I solve this problem?
Hello, I am a beginner at this and trying to go through your work as much precisely as possible. I wanted to recreate all your steps with one of your light field image and using your tar file. If i get successful I want to take images with our own plenoptic camera and recreate all the steps to get a clearer image.
I am getting error and can't really understand. Can you please help? adding the error message with this.
hello, I have a question I'd like to discuss with you. In your code, we can see that some parameters used in the disp2pts function are default parameters, but in fact for Illum, these parameters are not applicable, we need calibrate the LF camera, how to get the correct parameters according to the calibration results?
Hi again!
Hope you are doing well amidst all the recent happenings with the coronavirus outbreaks! I'm creating another issue here to follow up on the discussion last time on implementing focused plenoptics setup 😄
Being stuck at home these days, I finally got around to looking at the focused plenoptics setup on the papers you suggested! If I'm understanding it properly, spatial-angular resolution in the focused setup is determined by the ba
system instead from , where b is the distance between the sensor and the MLA, and a is the imaging distance of the microlens (aligned to the image formed by the main lens). Spatial resolution for the subaperture image would be the sensor resolution multiplied by (b/a), and angular resolution would be (a/b).
I found another software by Max, RayTracer
, that lets me simulate MLA images in the focused setup, at least that is what's claimed here. For his software, f-number matching is locked at a 95% ratio between the main lens and MLA.
The parameters for my mockup camera that I used to render the resulting synthetic images are as follows:
Sensor Resolution: 4096x3072 px
Pixel Size: 3.2 um
Main Lens Diameter: 7 mm
Main Lens F-Number: 16
Main Lens Focal Length: 112 mm
MLA Lens Diameter: 66 um (from my actual MLA film)
MLA F-Number: 15.15 (from my actual MLA film)
MLA Focal Length: 1 mm (from my actual MLA film)
Image Distance: 137.7 mm
Focus Distance: 600 mm
# "staircase" is placed ~3.5 m in front of camera
One thing I'm curious about, why might we want to do f-number matching for the MLA and main lens in a focused setup (as suggested by Lytro designs)? What if another f-number for the main lens is chosen?
But besides that, the real topic for this issue would be on how to go about adding a new feature on Plenopticam
to decode this focused plenoptics image.
It seams this is a reacquiring problem. I get this error with version 0.9.3.
Hi again!
While waiting for the other issue to be fixed, I have been looking over the other features within this library - are there any plans to add features for generating depth/disparity maps from a calibrated lightfield image? As I mentioned, I'm currently looking into a custom lightfield camera and trying to extract depth information from it.
If it will not be part of the scope of this library, how might I continue onward after getting a calibrated lightfield? I am not all too familiar with the typical approach to generating disparity from light fields, so any guidance on how to do so would be much appreciated!
On another note, another plenoptic library (plenpy
) that I've found has this feature implemented. The only issue with their library is that their generic lightfield camera implementation is incomplete... Would it be possible to export the calibrated lightfield data from this library and reload the lightfield on plenpy
?
Currently SciPy version 1.4.1 is required (introduced in this commit: 0d67816) which is a bit undesirable
Line 2 in 18f54f1
It would be great if this version requirement was removed - release 1.6.0 might have solved the issue.
In case the issue is still there.. is it an issue which could be fixed at this repository?
If you can provide some details about it @hahnec I might try to fix it. Cheers.
Hi, I'm trying to setup in order to test Lytro content on the LookingGlass Portrait.
I tried python3 -m pip install plenopticam
but ended up in dependency hell. After a bit of search I decided to use pyenv to rollback from Python 3.9 to 3.8 as apparently scipy had problem with it (no package yet). That ended up solving this part but then few more dependencies were still missing, namely sciki-image, which was fine, but also tk-tkinter which I didn't manage to solve :
File "/home/fabien/.pyenv/versions/3.8.12/lib/python3.8/tkinter/__init__.py", line 36, in <module>
import _tkinter # If this fails your Python may not be configured for Tk
As it gets a bit hairy trying to mix pyenv and system wide setups e.g sudo apt install python3-tk tk-dev
I'm wondering if a dockerfile wouldn't simplify. What I did for now was :
docker run -it --rm scipy/scipy-dev
python3.8 -m pip install scikit-image
python3.8 -m pip install plenopticam
plenopticam -h
which worked.
The type is .jpg, but it can't work
Plenopticam fails at Debayering stage on Ubuntu.
Already tried:
Ubuntu 18.04 with Python 3.8.10;
Ubuntu 20.04 with Pyhton 3.8.10;
Ubuntu 20.04 with Python 3.9;
Ubuntu 22.04 with Python 3.10.
Thank you
Hello,
The results are saved in a folder located at the same directory as the light field file.
I want to know how I can save them on a different directory.
Thank you
Hello, Can I use this code for my own LFM?
Hi, thanks for this project so I can try out real refocusing with light-field photographs.
It is painfully slow running on Windows and the CPU is certainly not fully used, is memory accessing the bottleneck in this case?
And the result is certainly much better comparing to Lytro Desktop in terms of accuracy. This makes my wonder the reason why Lytro Desktop didn't choose this way to process images? It seems to generate images by fusing preprocessed images with different depths.
Thanks again!
About the use of the Lytro Illum camera, I have a little question to ask you. Often, when I take a picture and look at it, there is an error mark (a cross), it cannot be displayed properly. What is the reason for this and how to avoid it? Looking forward to your reply!
Hi again!
While I'm working with the v0.3.0 test version, I notice that I get a circle P in the export images and .gifs when I am trying to process a set of lightfield images from a custom microlens camera, which is not the case when I was processing the provided example dataset.
What would this watermark mean in particular? Does it mean a bad lightfield image, an improper calibration process, failure to decompose, or something else? On other lightfield images tested, I also sometimes see green or black background images in the exported viewpoint stack.
I would like to test 'Plenopticam', but I don't have calibration data.
While searching on the Internet, I can only find lfp raw images without calibration data.
Would you let me know where can I get lfp raw images including calibration data?
It doesn't matter if the number of raw images is small.
Hi there,
I've just shared a writeup and a piece of code that enables direct access to all the features of the original Lytro camera.
You can find the writeup here https://github.com/ea/lytro_unlock and the code here https://github.com/ea/lytroctrl
Sharing in case somebody is still playing with these and has a cool idea that would benefit from easier control of the camera.
Cheers
I was trying to run the macos version, i downloaded the plenopticam_0.9.3.dmg.
the dmg opens fine, but trying to run the enclosed plenopticam.app gives an error saying it is damaged.
ive tried this on an i5 and m2 cpu.
any ideas?
hi,Hahne! I am very interested in your project. I run plenopticam under win, and when the program executes to light field alignment 47%, it doesn't move. Can I save sub-aperture images and how to save?
Hello!
I've recently been getting involved in researching application using plenoptic cameras and depth estimation, and I came across this Python library! I am trying to run the OpEx dataset, using the calibration and image .BMP file located at the root of the dataset directory (f197Inf9pxFinalShift12.7cmf22.bmp and f197Inf9pxFinalShift12.7cm.bmp), but I am getting an error about a NoneType for self._wht_img
.
(heka) kelvinchan@Quadro:~/Desktop/OpEx$ plenopticam -g
Plenopticam 0.2.0 by Christopher Hahne
Starting ...
Provided file /home/kelvinchan/Desktop/OpEx/f197Inf9pxFinalShift12.7cm/f197Inf9pxFinalShift12.7cm.json does not exist
White image file not present
Exception in Tkinter callback
Traceback (most recent call last):
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/tkinter/__init__.py", line 1702, in __call__
return self.func(*args)
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/tkinter/__init__.py", line 746, in callit
func(*args)
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 147, in poll
self.cur_thread.join()
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 341, in join
raise misc.errors.PlenopticamError(self.exc, cfg=self.cfg, sta=self.sta)
plenopticam.misc.errors.PlenopticamError: Traceback (most recent call last):
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 333, in run
self.ret = self._target(*self._args, **self._kwargs)
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/gui/widget_ctrl.py", line 256, in cal
cal_obj.main()
File "/home/kelvinchan/anaconda3/envs/heka/lib/python3.6/site-packages/plenopticam-0.2.0-py3.6.egg/plenopticam/lfp_calibrator/top_level.py", line 50, in main
self._wht_img = rgb2gray(self._wht_img) if len(self._wht_img.shape) is 3 else self._wht_img
AttributeError: 'NoneType' object has no attribute 'shape'
It also complains about the missing JSON for the calibration image, but that does not exist in this dataset. If this issue is due to the missing JSON, what are the values that Plenopticam expect in the calibration JSON?
Hi,
I just got a Lytro Illum and I'm starting doing some test using plenopticam.
For what I understand the lytro illum best resolution output is around 4Mpx after processing but the output of plenopticam is images of 622x432 pixels.
Is there a way to have a better resolution out of plenopticam ?
I'm guessing the setting "Micro image patch" size might be what I'm looking for but the description is a bit short for me understand fully.
Anyway, thanks to make this tool available. My final goal is to make a 3d scanner based on photos from the illum and plenopticam is a real good starting point for the pre-processing.
On raw images taken with a modified Nikon camera, in comp_bayer()
, the line a0 = (t0 << 4) + ((t1 & 0xF0) >> 4)
gives
(ValueError('operands could not be broadcast together with shapes (13408599,) (13408598,) '),)
i don't really get what bay_img is but if it's the bayer image, you could do self._bay_img = rawpy.imread(self._lfp_path).raw_image.astype('float')
innstead of calling self.comp_bayer()
. (It uses the rawpy library)
Does anyone know if this software works with the original Lytro Camera? The cuboid one.
I'm always getting this same error message, no matter if I select the Illum tar calibration file or the folder containing the calibration files. The correct white image files are all in there. I'm using an lfr image as my light field image. This is with version 0.8.0 and the Illum on the latest firmware version.
I still have a question, that is, the generated refocused images are generally darker. Is this normal? or have problems with my settings?
Originally posted by @hongyann in #2 (comment)
Hi
I got sub-apertures data, named by j_i.png
I would like to know what's the meaning of (j, i)
Can I explain 'j' as a vertical index of a camera position and 'i' as a horizontal index of a camera position?
sir,I want to measuring the dimensions of microscopic samples,like this http://lightfield-forum.com/wordpress/wp-content/uploads/2018/12/lightfield-feature-depth-mapping-distance-measurement-2.jpg,could you please tell me how to convert depth map to absolute distance? thank you very much
Is it possible to control (or estimate) the depth of field of the output images? e.g. is it possible to request an image that would be equivalent of a lens operating at f/8? or is it possible to estimate the equivalent f-stop of an output image?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.