ECCV 2024 Day4 - Oral: Point Clouds

 

 

 

ECCV 2024. 10. 02. Wednesday 
Oral 3C: Point Clouds

 

 

 

Rethinking Data Augmentation for Robust LiDAR Semantic Segmentation in Adverse Weather

Junsung Park · Kyungmin Kim · Hyunjung Shim

 

 

 

☁️

예전에 윤영이랑 같이 이야기하던 내용이 나오길래 집중해서 들었다.!

날씨가 좋지 않은 상황에서도 (데이터셋 관점), 잘 동작하는 모델을 학습시키기 위한 방법

 

 

 

 

 

 

Abstract

 Existing LiDAR semantic segmentation methods commonly face performance declines in adverse weather conditions. Prior research has addressed this issue by simulating adverse weather or employing universal data augmentation during training. However, these methods lack a detailed analysis and understanding of how adverse weather negatively affects LiDAR semantic segmentation performance.

 Motivated by this issue, we characterized adverse weather in several factors and conducted a toy experiment to identify the main factors causing performance degradation: (1) Geometric perturbation due to refraction caused by fog or droplet in the air and (2) Point drop due to energy absorption and occlusions.Based on this analysis, we propose new strategic data augmentation techniques. Specifically, we first introduced a Selective Jittering (SJ) that jitters points in the random range of depth (or angle) to mimic geometric perturbation. Additionally, we developed a Learnable Point Drop (LPD) to learn vulnerable erase patterns with Deep Q-Learning Network to approximate point drop phenomenon from adverse weather conditions.

 Without precise weather simulation, these techniques strengthen the LiDAR semantic segmentation model by exposing it to vulnerable conditions identified by our data-centric analysis. Experimental results confirmed the suitability of the proposed data augmentation methods for enhancing robustness against adverse weather conditions. Our method attains a remarkable 39.5 mIoU on the SemanticKITTI-to-SemanticSTF benchmark, surpassing the previous state-of-the-art by over 5.4%p, tripling the improvement over the baseline compared to previous methods achieved.

 

 

 

Problem - 현재 방법의 한계점

 

 

 

deterioration의 main factor는 무엇일까?

 

 

 

결론 - Geometric Perturbation and Point Drop

 

 

 

Methods

 

 

 

Experiments

 

 

 

 

 

 

 

 

 

🔗 Git-hub Link

https://github.com/engineerJPark/LiDARWeather

GitHub - engineerJPark/LiDARWeather: [ECCV 2024 Oral] Official code of "Rethinking Data Augmentation for Robust LiDAR Semantic S

 

 

 

 

 

RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentation

Luis Li · Hubert P. H. Shum · Toby P Breckon

 

 

 

 

 

 

Abstract

 3D point clouds play a pivotal role in outdoor scene perception, especially in the context of autonomous driving. Recent advancements in 3D LiDAR segmentation often focus intensely on the spatial positioning and distribution of points for accurate segmentation. However, these methods, while robust in variable conditions, encounter challenges due to sole reliance on coordinates and point intensity, leading to poor isometric invariance and suboptimal segmentation.

 To tackle this challenge, our work introduces Range-Aware Pointwise Distance Distribution (RAPiD) features and the associated RAPiD-Seg architecture. Our RAPiD features exhibit rigid transformation invariance and effectively adapt to variations in point density, with a design focus on capturing the localized geometry of neighboring structures. They utilize inherent LiDAR isotropic radiation and semantic categorization for enhanced local representation and computational efficiency, while incorporating a 4D distance metric that integrates geometric and surface material reflectivity for improved semantic segmentation.

 To effectively embed high-dimensional RAPiD features, we propose a double-nested autoencoder structure with a novel class-aware embedding objective to encode high-dimensional features into manageable voxel-wise embeddings. Additionally, we propose RAPiD-Seg which incorporates a channel-wise attention fusion and a two-stage training strategy, further optimizing the embedding for enhanced performance and generalization. Our method outperforms contemporary LiDAR segmentation work in terms of mIoU on SemanticKITTI (76.1) and nuScenes (83.6) datasets (leaderboard rankings: 1st on both datasets).

 

 

 

해석하는 방식이 엄청 흥미로웠다.!

 

 

 

 

 

 

Future Work

 

 

 

🔗 Git-hub Link

https://github.com/l1997i/Rapid_Seg

GitHub - l1997i/Rapid_Seg: 🔥(ECCV 2024 Oral) RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Seg