Many of today\u2019s self-driving cars \u201csee\u201d the road through two senses at once. Cameras capture color, texture and the outline of a pedestrian, while light detection and ranging (LiDAR) fires laser pulses to measure exactly how far away that pedestrian stands. Together, they provide complementary information and outperform either sensor alone.<\/p>\n
This pairing has a weakness, however. Trained mostly on clear daytime footage, the fusion model learns to lean on whichever sensor performs best there and may fail in challenging scenarios. Rain can scatter LiDAR beams, for instance, while a night drive leaves cameras battling darkness and glare.<\/p>\n
To address this, assistant professor Zhao Na at Singapore University of Technology and Design (SUTD) led two studies that built different models. In \u201cCCF: Complementary collaborative fusion for domain generalized multi-modal 3D object detection<\/a>,\u201d the research team rebalances how a detector weighs its sensors so it holds up in new surroundings. In \u201cPanDA: Unsupervised domain adaptation for multimodal 3D panoptic segmentation in autonomous driving<\/a>,\u201d they harness unlabeled data from a new environment to help the model adapt once it arrives.<\/p>\n","protected":false},"excerpt":{"rendered":" Many of today\u2019s self-driving cars \u201csee\u201d the road through two senses at once. Cameras capture color, texture and the outline of a pedestrian, while light detection and ranging (LiDAR) fires laser pulses to measure exactly how far away that pedestrian stands. Together, they provide complementary information and outperform either sensor alone. This pairing has a [\u2026]<\/p>\n","protected":false},"author":662,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,1491],"tags":[],"class_list":["post-246913","post","type-post","status-publish","format-standard","hentry","category-robotics-ai","category-transportation"],"_links":{"self":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/246913","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/users\/662"}],"replies":[{"embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/comments?post=246913"}],"version-history":[{"count":0,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/posts\/246913\/revisions"}],"wp:attachment":[{"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/media?parent=246913"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/categories?post=246913"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lifeboat.com\/blog\/wp-json\/wp\/v2\/tags?post=246913"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}