Ex-Two Sigma, ex-Trello, ex-Stitch Fix. Building DAGWorks, Inc. (YC W23)
Bay Area, CA
Joined January 2013
- Tweets2.7K
- Following346
- Followers261
- Likes35.7K
Pinned Tweet
Do you always need an #orchestrator?
Thrilled to share the work we did with @DonnyGreenberg at @runhouse_, showing how "unbundling" the orchestrator into components can build a more flexible system to track assets/manage heterogenous #ml/#ai infra.
blog.dagworks.io/p/lean-data…
elijah retweeted
People have asked me can @hamilton_os interoperate on @SnowflakeDB ? The answer is yes, and I'm this simple example we show that the Hamilton UI can too! – at San Francisco, CA
Super excited for this HOW-TO on running the Hamilton
UI on @SnowflakeDB :
"Observability of Python Code and Application Logic with @hamilton_os UI on @SnowflakeDB Containers" Services by Greg Przemyslaw Kantyka link.medium.com/FIwGhJqzsPb
elijah retweeted
Super excited for this HOW-TO on running the Hamilton
UI on @SnowflakeDB :
"Observability of Python Code and Application Logic with @hamilton_os UI on @SnowflakeDB Containers" Services by Greg Przemyslaw Kantyka link.medium.com/FIwGhJqzsPb
elijah retweeted
Super excited for this post by @thierryjean that provides a well laid out golden path for folks iterating in the #GenAI space.
This follows up from my tweet yesterday about framework coupling. If the framework allows loose coupling, then graduating out of it isn't a thing you'll run into. In this post Thierry lays out to use Burr and how it doesn't get in your way as you build more complex and sophisticated workflows and agents.
blog.dagworks.io/p/the-gradu…
A @burr_framework user shared this video youtube.com/watch?v=evmZTh7l… with me on the "problem with frameworks". Worth a watch.
I 💯% agree with the take home. The "coupling problem" is the reason why people graduate out of frameworks like @langchain & @llama_index. You become overly coupled and then find yourself overtime asking why am I using this? It's not that the framework authors did this maliciously, it's just that they optimized for their needs, in this case for POCs, which is not the same as long lived maintainable code.
That's why we see people either build their own frameworks, or they hopefully discover @hamilton_os & @burr_framework and realize that there are frameworks that allow "loose coupling" that don't get in your way (as a platform person and framework author we optimize for maintainable code). A good example of this is @FastAPI - very easy to not couple your business logic and iterate quickly no matter the age of the code.
So how can you tell where a framework lies on the spectrum of tight coupling versus not? Some tests:
(1) How many "objects" do I need to use from the framework to couple the logic I want to happen? The fewer the better. E.g. with @hamilton_os by default it's 0. With @burr_framework it's just 1 and that's due to state for edge transitions, but that's easy to decouple from.
(2) How many framework imports do you need to get something to run? The less the better - consider @burr_framework vs @langchain using #langgraph (see images)
If you're feeling framework pains, or are curious on how it can be done right, checkout:
github.com/DAGWorks-Inc/burr (agents) github.com/DAGWorks-Inc/hami… (pipelines/tools) or subscribe to our blog - blog.dagworks.io/
#python #opensource #genai #llmops
elijah retweeted
Yep that's why @hamilton_os and @burr_framework observability is built in & pluggable by design, so you can offer all the trappings for enterprise without having to engineer for it later; graph / flowchart mental models FTW!
elijah retweeted
Elijah Ben Izzy (@elijahbenizzy) talks AI hype, streaming chatbots, and the future of energy consumption in tech on ep. 22 of #Generationship. Join host @rachelchalmers for this fascinating deep dive on the intersection of AI and infrastructure.
hubs.ly/Q02T9PgQ0
elijah retweeted
FlowerPower - Python workflow framework
Based on @hamilton_os and APScheduler
🔄 DAG-based pipelines
⏰ Scheduling
⚙️ Simple YAML-based setup
📊 Tracking
📡 Distributed execution
github.com/legout/flowerpowe…
elijah retweeted
Diving into OSS is a blast and super important!
Lately, I've been all about @burr_framework for whipping up state machines and AI agents.
Just dropped my first commit in the new release 🎉
Give BURR a look guys, it plays nice with any framework. AI devs, you'll love it!
#OpenSource #AI
elijah retweeted
In Episode 22 of Generationship, @RachelChalmers chats with Elijah Ben Izzy (@elijahbenizzy) of @DagWorks about Burr (@burr_framework), a tool that’s revolutionizing AI workflows. Tune in!
hubs.ly/Q02T9RF40
elijah retweeted
In ep. 22 of Generationship, @rachelchalmers speaks with Elijah Ben Izzy (@elijahbenizzy) of @DagWorks about the importance of streaming in chatbot interactions, highlighting how partial, real-time responses can significantly enhance user engagement.
hubs.ly/Q02T9RdP0
elijah retweeted
Guess who is on the #instructor blog? ME!
The TLDR:
- Use #burr and #instructor to ship more quickly
- Instructor for structuring LLM interactions
- Burr for app structure & observability
In this example we create flashcards from @YouTube videos.
python.useinstructor.com/blo…
elijah retweeted
Super excited to see this blog published by @jxnlco & Thierry Jean.
@burr_framework is lightweight & very agnostic to what you use with it.
Instructor is also lightweight.
Together: they create a fast means to iterate and ship something reliable.
Guess who is on the #instructor blog? ME!
The TLDR:
- Use #burr and #instructor to ship more quickly
- Instructor for structuring LLM interactions
- Burr for app structure & observability
In this example we create flashcards from @YouTube videos.
python.useinstructor.com/blo…
Really excited to have @sh_reya on board!
🎉 Excited to announce that @sh_reya has joined us @DagWorks as an advisor. Her research & what we're building with @hamilton_os & @burr_framework overlaps. I'm excited to work together more closely.
For more see my update: new features, docs, video etc
linkedin.com/pulse/week-octo…
Stoked to be on, had a great conversation! Thanks @heavybit for having me.
Check out ep. 22 of Generationship! Elijah Ben Izzy (@elijahbenizzy) of @DagWorks unpacks his work with AI tools, the importance of user engagement in chatbot responses, and the future of AI in the context of energy consumption and hype.
hubs.ly/Q02TC_P80
elijah retweeted
Check out ep. 22 of Generationship! Elijah Ben Izzy (@elijahbenizzy) of @DagWorks unpacks his work with AI tools, the importance of user engagement in chatbot responses, and the future of AI in the context of energy consumption and hype.
hubs.ly/Q02TC_P80
elijah retweeted
My current favorite OSS is Burr.
If you’re building a gen-AI, remember that the production-ready system will operate as a state machine.
The @burr_framework excels in this area.
elijah retweeted
Check out the latest article in my newsletter: Week of October 7th
Some things that shipped:
@burr_framework:
- V1.0 of our #annotations workflow. Take it for a spin, and let us know what needs to be improved!
@hamilton_os:
- new "mutate" decorator. Helpful to more quickly update and modify "funcitons/nodes"
- updated @SlackHQ notifier plugin; get errors more nicely formatted.
Otherwise new blogs, and the Hamilton meet-up is next week. Links in the newsletter!
linkedin.com/pulse/week-octo… via @LinkedIn
#python #opensource #llmops #mlops #datascience #dataengineering #machinelearning #genai #genaiops
elijah retweeted
@burr_framework Office hours happening now (9:30am PT for 60mins) -- come chat and ask questions. Join @discord for the link - discord.gg/KtfH3dykab
elijah retweeted
I'm really excited to demo @hamilton_os & @burr_framework at the #SF Awesome #AI Dev Tools event by @yujian_tang tonight (10/7) hosted by from @OSS4AI. It's just one of the many #SFTechWeek events.
Come by and chat! RSVP here: lu.ma/0mh3j8m1
#python #opensource #aidevtools
