📍 𝗖𝗮𝗻 𝗔𝗜 𝗡𝗮𝘃𝗶𝗴𝗮𝘁𝗲 𝗠𝗮𝗽𝘀 𝗟𝗶𝗸𝗲 𝗛𝘂𝗺𝗮𝗻𝘀 𝗗𝗼? 𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗠𝗮𝗽𝗕𝗲𝗻𝗰𝗵! 🗺️🤖 𝘙𝘦𝘢𝘥𝘪𝘯𝘨 𝘮𝘢𝘱𝘴, like Google Maps and Theme Park Maps, is second nature for humans. It is a highly challenging task that requires visual understanding, spatial reasoning, and long-horizon planning. We're curious - 𝗖𝗮𝗻 𝗟𝗮𝗿𝗴𝗲 𝗩𝗶𝘀𝗶𝗼𝗻-𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 (𝗟𝗩𝗟𝗠𝘀) 𝗱𝗼 𝗶𝘁 𝘁𝗼𝗼? 🤔 We’re excited to share 𝗠𝗮𝗽𝗕𝗲𝗻𝗰𝗵, the first-ever dataset and benchmark specifically designed for evaluating how well LVLMs perform on pixel-based map navigation tasks! 🚀 🔑 𝗪𝗵𝘆 𝗠𝗮𝗽𝗕𝗲𝗻𝗰𝗵 𝗶𝘀 𝗮 𝗚𝗮𝗺𝗲-𝗖𝗵𝗮𝗻𝗴𝗲𝗿: • 📌 1600+ Complex Pathfinding Queries from 100 uniquely challenging map scenarios (urban areas, theme parks, universities, malls, and more). • 📌 Introduces Map Space Scene Graph (MSSG): a novel data structure for mapping visual landmarks and spatial relationships to structured navigation tasks. • 📌 Evaluates state-of-the-art LVLMs like GPT-4o, Llama-3.2, and Qwen-2-VL under zero-shot and Chain-of-Thought (CoT) reasoning methods, revealing key insights into their spatial reasoning and navigation abilities. 🚩 𝗞𝗲𝘆 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀: • Despite their impressive capabilities, current LVLMs struggle significantly with spatial reasoning and structured decision-making. • CoT prompting boosts spatial reasoning performance but sometimes introduces redundant details. 👀 𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝗼𝘂𝗿 𝗳𝗶𝗻𝗱𝗶𝗻𝗴𝘀, 𝗱𝗮𝘁𝗮𝘀𝗲𝘁, 𝗮𝗻𝗱 𝗰𝗼𝗱𝗲 𝗵𝗲𝗿𝗲: 🔗 Arxiv: lnkd.in/gBv-sFJ3 Huge thanks to our incredible collaborators for making this happen, from @TAMU, @UCBerkeley, @mbzuai, @UMich, and @UCRiverside! 🎉 Let’s continue to bridge the gap between human intuition and AI navigation! 🗺️💡