Comments (6)
Hi, @stephan000
Thank you for your interest in this project. Class 0 means unoccupied.
I wonder when you visualize the estimated semantic map, do you see any occupied areas?
If so, there might be something incorrect in the evaluation code. I will look into it and let you know.
from bkisemanticmapping.
Thank you for your prompt reply!
The estimated semantic map seems to be somewhat better qualitatively.
However, most estimated classes are unoccupied.
000000.txt
The occupied threshold is 0.7, which is the default value in config/method/semantic_bki.yaml
.
So, Ι think this value is fine.
from bkisemanticmapping.
Hi @stephan000,
As the qualitative result looks fine to me. I don't think there's an issue with the code.
The quantitative result given in the paper is obtained for all the semanticKITTI sequences, so I am not sure if the result you got is correct or not.
Also, the 0.7 threshold is not used in this method, the unoccupied voxel is determined by the estimated Categorical distribution.
from bkisemanticmapping.
Hi @ganlumomo ,
To my understanding, the quantitative results listed in the README show the IoU values per scene.
Therefore, I expected that running semantickitti_04, which you provided, would give me the S-BKI results for sequence 04.
Is this incorrect?
It is enough for me to check if semantickitti_04 gives me the right result. However, the results I am getting now seem to be much poorer than the results in the paper.
My environment can be reproduced by docker. If there is a mistake in the procedure, please let me know.
FROM ros:kinetic
ENV DEBIAN_FRONTEND=noninteractive
RUN apt update -y -qq && \
apt upgrade -y -qq --no-install-recommends && \
apt install -y -qq \
wget \
curl \
unzip \
ros-kinetic-tf-conversions \
ros-kinetic-vision-opencv \
ros-kinetic-pcl-ros \
ros-kinetic-rviz
ENV CATKIN_WS=/workspace/catkin_ws
WORKDIR ${CATKIN_WS}/src
RUN git clone https://github.com/ganlumomo/BKISemanticMapping && \
cd ${CATKIN_WS} && \
. /opt/ros/kinetic/setup.sh && \
/opt/ros/kinetic/bin/catkin_make
ARG FILE_ID=19Dv1jQqf-VGKS2qvbygFlUzQoSvu17E5
ARG FILE_NAME=semantickitti_04.zip
RUN cd ${CATKIN_WS}/src/BKISemanticMapping/data && \
curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=${FILE_ID}" > /dev/null && \
CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)" && \
curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=${FILE_ID}" -o ${FILE_NAME} && \
unzip ${FILE_NAME} && \
rm ${FILE_NAME}
RUN set -x && \
cd ${CATKIN_WS}/src/BKISemanticMapping/ && \
sed -i -e "s/^scan_num: 100/scan_num: 271/g" config/datasets/semantickitti.yaml && \
curl https://bootstrap.pypa.io/pip/3.5/get-pip.py -o get-pip.py && \
python3 get-pip.py && \
rm get-pip.py && \
python3 -m pip install jupyter numpy scikit-learn && \
jupyter nbconvert --to python semantickitti_evaluation.ipynb && \
sed -i -e "s/^evaluation_folder.*/evaluation_folder\ =\ '.\/data\/semantickitti_04\/evaluations\/'/g" semantickitti_evaluation.py
Please run the following commands:
docker build -t bkisemanticmapping .
docker run -dit --name bki_sem_map bkisemanticmapping
docker exec -it bki_sem_map bash
source /workspace/catkin_ws/devel/setup.bash
roslaunch semantic_bki semantickitti_node.launch
cd /workspace/catkin_ws/src/BKISemanticMapping
python3 semantickitti_evaluation.py
from bkisemanticmapping.
Hi @stephan000
I just fixed the bug in the commit: 1e18b1e. Apparently, I added a fixed transformation to the map for a better visualization view, but somehow forgot to add the same transformation in the evaluation code.
Sorry for the inconvenience. Also, the results in the paper are obtained using resolution of 0.1. So you might need to change the resolution here: BKISemanticMapping/config/datasets/semantickitti.yaml.
from bkisemanticmapping.
Hi @ganlumomo
I got almost the same results as in the paper with the default resolution.
Thanks for your quick response!
from bkisemanticmapping.
Related Issues (3)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bkisemanticmapping.