Reconstructing the world one image at a time at https://xitter.cf/t.co/AsIUkJrfBC prev @RealityLabs and @mapillary
Zurich
Joined November 2010
- Tweets523
- Following306
- Followers258
- Likes460
Pau Gargallo retweeted
Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀
One universal model enables SoTA for:
🔥 Mono Depth Estimation
🔥 Multi-View SfM
🔥 Multi-View Stereo
🔥 Depth Completion
🔥 Registration
… and many more possibilities! – plus everything is metric 🎯
We release code for data processing, training, benchmarking & ablations – everything Apache 2.0!
Details & Links 👇
You know you are an old CV fart when you see SFM and think "Structure from Motion", but they mean "Spatial Foundation Models"
Cool stuff: spaitial.ai/#Mission
Pau Gargallo retweeted
Gemini + π0 = actually useful robots! (Similar to what @physical_int did with "Hi Robot")
I can now verbally tell the robot that I'm building a red Lego wall or wooden tower, and it will infer the next steps by itself and pass me the necessary pieces, tools, or materials, ha!
You can also just ask it to bring you things!
The pipeline works as follows:
- OpenAI Whisper (local) → speech to text
- Gemini → makes sense of user requests, converts to robot tasks, bounding boxes, grasping points, etc. (System 2 thinking FTW!)
- π0 → robotic actions
The π0 was finetuned just for pick-and-place Lego bricks only, and it generalizes beautifully to all kinds of tasks. However, there's lots of room for improvement when it comes to grasping & accuracy.
Things that could help:
- Conditioning on grasping points
- Better data collection (I'm not that great at teleop)
- Lots more synthetic data and simulations
Pau Gargallo retweeted
Thought about generating realistic 3D urban neighbourhoods from maps, dawn to dusk, rain or shine? Putting heavy snow on the streets of Barcelona? Or making Paris look like NYC? We built a Streetscapes system that does all these. See boyangdeng.com/streetscapes. (Showreel w/ 🔊 ↓)
Pau Gargallo retweeted
The wait is over 📢 MAST3R is out! DUSt3R+ dense local feature maps & metric depth - 1st in #MapFreeReloc leaderboard, can handle 1000s of images 😀 !!
Blog: shorturl.at/9JTH2
Code: github.com/naver/mast3r
Paper: arxiv.org/abs/2406.09756
Pau Gargallo retweeted
The Quest v64 update brought two undocumented major new features: furniture recognition on Quest 3 and simultaneous hands & controllers in the home space:
uploadvr.com/quest-v64-undoc…
Pau Gargallo retweeted
Now available on Mapillary: Neural Radiance Fields (NeRFs)! 🎊
NeRFs allow for the transformation of a collection of 2D images into detailed, immersive 3D reconstructions.
Read our blog post to learn more and see how you can get started with NeRFs: blog.mapillary.com/update/20…
Pau Gargallo retweeted
(1/2)
We've won an Oscar!!
Dolby Atmos has received the Academy Plaque for Scientific & Technical Achievements, the highest award short of an Oscar statuette, by the Academy of Motion Picture Arts & Science.
youtu.be/h3IyGHbVlMw
Pau Gargallo retweeted
oh man it's cool when math works and cameras work and digital things sort of match up with the real world
i took photos at Hetch Hetchy over the summer, ran opensfm, and the geometry of the reservoir shore actually kind of lines up at a larger scale than i was expecting!
Pau Gargallo retweeted
This is my favourite part of the exhibition! Glad you let your sponsor show off their plans to reduce carbon emissions
Pau Gargallo retweeted
Uncertainties are everywhere in robotics and are fundamental for probabilistic methods such as factor graphs. However, frames, conventions, and other concepts can make things confusing. I wrote 3 posts for gtsam.org invited by @fdellaert to explain this.
Pau Gargallo retweeted
BIG NEWS: today we’re releasing the Mapillary Street-Level Sequences Dataset, the world’s most diverse publicly available dataset for training place recognition algorithms. Read more below! 🎉 ow.ly/uFSu50zpDKw
Pau Gargallo retweeted
John Conway, the creator of the Game of Life passed away last night. So many hours coding and playing with one of his many ideas. Here is an implementation I did in WebGL several years ago of the Game of Life: tamats.com/apps/conway/ See you in a simpler universe.
Pau Gargallo retweeted
Here is a better-quality video clip of John H Conway performing the knot trick that @wtgowers mentioned.
It’s impressively done. Even watching it on a loop, I can’t see the sleight of hand.
Pau Gargallo retweeted
This is the current world record of distance landscape photography: the Alps as seen from of the Pyrenees ow.ly/gZpk50yYUpk
Pau Gargallo retweeted
Jailing your citizens for political protest makes everything worse. It is what happens in authoritarian nations. It will contribute to more EU fracture if Spain doesn't act like a democracy. axios.com/catalan-protesters…
Pau Gargallo retweeted
I have no right to comment on Catalan independence. It is for the Catalans to decide. BUT, the rest of us must rise up against politicians being sentenced to long prison stretches in the heart of Europe for pursuing political agendas mandated by voters
theguardian.com/world/2019/o…
