Slava Elizarov (@DoctorDukeGonzo) profile picture

Staff Research Scientist @canva, ex-Unity | Generative models, Computer Graphics

Germany
Joined April 2012
Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)Photo by Slava Elizarov (@DoctorDukeGonzo)
Does 3D generation always have to be either slow or complex and data-hungry?🤔 We don’t think so! With Geometry Image Diffusion, we’re all about reusing (and recycling ♻️) what already works — making it faster and easier by reducing complexity and data needs 🚀(1/10)
19
60
7
448
67,915
Slava Elizarov retweeted
Have you ever gotten tired of boring plain linear layers and wanted a more complex function? We find that attaching low rank nonlinear residual functions can significantly accelerate pretraining, with an identified variant, CosNet, consistently observing 20+% wallclock speedup!
16
46
5
262
43,134
Thrilled to share that Geometry Image Diffusion has been accepted to #ICLR2025! 🚀 Paper: openreview.net/forum?id=Glm7…
Does 3D generation always have to be either slow or complex and data-hungry?🤔 We don’t think so! With Geometry Image Diffusion, we’re all about reusing (and recycling ♻️) what already works — making it faster and easier by reducing complexity and data needs 🚀(1/10)
1
10
78
5,131
I’m currently exploring new job opportunities🧑‍🔬 My work revolves around text-to-3D with Geometry Images, generative UV mapping, multi-view models for texturing, and other genAI applications in graphics. I’d love to discuss how I can contribute to your research efforts!
Does 3D generation always have to be either slow or complex and data-hungry?🤔 We don’t think so! With Geometry Image Diffusion, we’re all about reusing (and recycling ♻️) what already works — making it faster and easier by reducing complexity and data needs 🚀(1/10)
4
10
1
46
9,869
P.S. Thanks to @CiaraRowles1, Simon Donné, @esx2ve, @danteCIM, and @bostadynamics for being such an awesome team! Additional thanks to @xdralex and Dr. Lev Melnikovsky from the Weizmann Institute for all the insightful discussions we had during this project
1
8
2,312
P.P.S. We recommend you check out Omages (omages.github.io) by @yan_xg , an awesome concurrent work that also explores geometry images (called "Omages") for 3D generation. We believe GIMs have a bright future in deep learning — let’s bring it forward together 🚀
1
1
10
1,628
Slava Elizarov retweeted
We're excited to release our new research paper: IP Adapter Instruct: Resolving Ambiguity in Image-based Conditioning using Instruct Prompts arxiv: arxiv.org/abs/2408.03209 project page (with live demo!): unity-research.github.io/IP-…
17
32
3
171
13,552
IPAdapter-Instruct, a new release led by @CiaraRowles1 @unitygames Feed the IPAdapter image and instruct the model what to use from it: style, color, composition, pose, face! SD1.5, XL and 3 support 🔥 ✴️ Weights: huggingface.co/CiaraRowles/I… ▶️ Demo: huggingface.co/spaces/unity/… 👩‍💻 Code: github.com/unity-research/IP…
Unity presents IPAdapter-Instruct Resolving Ambiguity in Image-based Conditioning using Instruct Prompts discuss: huggingface.co/papers/2408.0… Diffusion models continuously push the boundary of state-of-the-art image generation, but the process is hard to control with any nuance: practice proves that textual prompts are inadequate for accurately describing image style or fine structural details (such as faces). ControlNet and IPAdapter address this shortcoming by conditioning the generative process on imagery instead, but each individual instance is limited to modeling a single conditional posterior: for practical use-cases, where multiple different posteriors are desired within the same workflow, training and using multiple adapters is cumbersome. We propose IPAdapter-Instruct, which combines natural-image conditioning with ``Instruct'' prompts to swap between interpretations for the same conditioning image: style transfer, object extraction, both, or something else still? IPAdapterInstruct efficiently learns multiple tasks with minimal loss in quality compared to dedicated per-task models.
3
27
3
102
19,198
Slava Elizarov retweeted
Unity presents IPAdapter-Instruct Resolving Ambiguity in Image-based Conditioning using Instruct Prompts discuss: huggingface.co/papers/2408.0… Diffusion models continuously push the boundary of state-of-the-art image generation, but the process is hard to control with any nuance: practice proves that textual prompts are inadequate for accurately describing image style or fine structural details (such as faces). ControlNet and IPAdapter address this shortcoming by conditioning the generative process on imagery instead, but each individual instance is limited to modeling a single conditional posterior: for practical use-cases, where multiple different posteriors are desired within the same workflow, training and using multiple adapters is cumbersome. We propose IPAdapter-Instruct, which combines natural-image conditioning with ``Instruct'' prompts to swap between interpretations for the same conditioning image: style transfer, object extraction, both, or something else still? IPAdapterInstruct efficiently learns multiple tasks with minimal loss in quality compared to dedicated per-task models.
2
33
6
150
54,785