File over app for AI https://xitter.cf/t.co/vVu3KrixaW https://xitter.cf/t.co/Ok61BXZIW1 https://xitter.cf/t.co/7vOO03PmQ8 youtube: loganhallucinates
PNW
Joined November 2009
- Tweets3.6K
- Following622
- Followers2.2K
- Likes17.8K
Logan Yang retweeted
A year ago the question was which model. Now it's which harness.
Pi, Exo, Claude Code, Codex, DeepSeek Harness and 4 others. Same model, same tasks, same runtime. 360 runs, 2 billion tokens.
Pass rates: 50% to 67%.
Cost per pass: $1.05 to $18.34.
Introducing FrontierHarness Eval. 🧵
Logan Yang retweeted
we built pdf-inspector so agents can process PDFs without waiting on OCR. it classifies any PDF in ~20ms and extracts clean markdown locally
→ 200 PDFs processed in 2.8s
→ top quality in extracting tables + graphs
→ built in rust
→ open source
github.com/firecrawl/pdf-ins…
Logan Yang retweeted
The first wave of AI-native applications is wrapping tokens and providing in-app agents.
As agent usage centralizes around core apps (e.g. Claude Code, Codex), there's this emerging wave of building software that doesn't need to have AI on its own, but is extremely easy to plug into whatever the leading AI app is.
Open question on which patterns will win out
Logan Yang retweeted
In good hands
There is a feeling I search for: being in good hands. It is the feeling I look to give and the feeling I look to receive.
I know I am in good hands when I sense a cohesive point of view expressed with attention to detail.
I can feel it almost instantly. In any medium. Music, film, fashion, architecture, writing, software. At a Japanese restaurant it's what omakase aims to be. I leave it up to you, chef.
When I am in good hands I open myself to a state of curiosity and appreciation. I allow myself to suspend preconceived notions. I give you freedom to take me where you want to go. I immerse myself in your worldview and pause judgement.
I want to be convinced of something new. I want my mind to be changed. Later I may disagree, but for now I am letting the experience soak in.
That trust doesn't come easily. As an audience member it's about feeling cared for from the moment I interact with your work. It's about feeling a well-defined point of view permeate what you make.
If my mind was changed, I must have been in good hands.
ALT There is a feeling I search for: being in good hands. It is the feeling I look to give and the feeling I look to receive. I know I am in good hands when I sense a cohesive point of view expressed with attention to detail. I can feel it almost instantly. In any medium. Music, film, fashion, architecture, writing, software. At a Japanese restaurant it's what omakase aims to be. I leave it up to you, chef. When I am in good hands I open myself to a state of curiosity and appreciation. I allow myself to suspend preconceived notions. I give you freedom to take me where you want to go. I immerse myself in your worldview and pause judgement. I want to be convinced of something new. I want my mind to be changed. Later I may disagree, but for now I am letting the experience soak in. That trust doesn't come easily. As an audience member it's about feeling cared for from the moment I interact with your work. It's about feeling a well-defined point of view permeate what you make.
Logan Yang retweeted
🚀 Want FREE models you can plug into OpenClaw or Hermes?
Here are 9 resources you can use for free access to model APIs
No local setup, no credit card, just pure cloud APIs with OpenAI-compatible endpoints
You can’t get free Opus quality (yet) but all of these have genuine free tiers right now (rate limits may apply) and are good enough to get started if you don’t want to spend $ to get started with agents
1️⃣ OpenRouter Free Models
(Gemma 4 31B/26B, NVIDIA Nemotron 3 Super 120B MoE, MiniMax M2.5, Qwen3 variants, Llama 4/3.3, gpt-oss-120B, Arcee Trinity, etc.)
• ~29 completely free $0/M token models
• Insane variety + top-tier open model evals (especially coding & agents)
• Best for rotating models automatically
👉 Sign up: openrouter.ai/keys
2️⃣ Google Gemini API
(Gemini 2.5 Pro / Flash series)
• Strongest overall free frontier model
• Excellent multimodal, 1M+ context, native tool calling & agentic performance
• Very generous free limits (often 5–15 RPM)
👉 Sign up: aistudio.google.com/app/apik…
3️⃣ NVIDIA
(Nemotron variants, Llama 3.3 70B, Qwen3 235B, Mistral Large, etc.)
• Optimized high-performance open models
• Free prototyping tier (~40 RPM)
👉 Sign up: build.nvidia.com/explore/dis…
4️⃣ Grok Cloud
(Llama 4 Scout, Llama 3.3 70B, Qwen3 32B, gpt-oss models, etc.)
• Blazing-fast inference (hundreds of tokens/sec)
• Perfect for real-time agents
• Strong open-model performance with solid free tier
👉 Sign up: console.groq.com/keys
5️⃣ Cerebras Cloud
(Qwen3 235B, Llama 3.3 70B, DeepSeek variants, etc.)
• Massive models with excellent reasoning/coding evals
• Very generous daily free limits (~30 RPM, up to 1M+ tokens/day on some)
👉 Sign up: cloud.cerebras.ai/
6️⃣ Mistral La Plateforme
(Mistral Large 3, Small 3.1, Ministral 8B, etc.)
• Strong in coding, multilingual & agentic tasks
• Solid free tier (~1 req/s, ~1B tokens/month)
👉 Sign up: console.mistral.ai/api-keys
7️⃣ Cohere
(Command A, Command R+, Aya Expanse 32B, etc.)
• Free tier: 20 RPM, 1K requests/month
👉 Sign up: dashboard.cohere.com/api-key…
8️⃣ GitHub Models
(Llama 3.3 70B, DeepSeek R1, some GPT-4o previews, etc.)
• Decent mid-tier evals with easy GitHub integration
• Free tier limits (10–15 RPM)
👉 Sign up: github.com/marketplace/model…
9️⃣ Cloudflare Workers AI
(Llama 3.3 70B, Qwen QwQ 32B, etc.)
• Lightweight but solid for simple agents
• Free tier: 10K neurons/day
👉 Sign up: dash.cloudflare.com/profile/…
Pro tips for agent builders:
• Most work instantly with OpenAI SDK (just change base URL + your key)
• Start with OpenRouter for quality/variety (they often feature new free models)
• Add Groq as speed fallback
• Rotate providers when you hit caps
Free intelligence for your agent is just a signup away!
Logan Yang retweeted
after turboquant and qwen3.5-35b-a3b, i got curious: how realistic is it to use kv cache as a document store today? to have vectorless, RAG-less search. so i prefilled 258K out of 262K context window on L4 (a budget GPU popular in prod). ~99% of the slot is pre-computed and stored, users load it on the fly in ~1s. system prompt + query append to the end, generation takes ~3K tokens, enough for search. at 99% fill rate, decoding runs ~20 tps on L4.
i prepared some ego datasets (jina papers, which i know best), plus popular novels in chinese and english. the results are actually pretty good. some hallucination, but most answers are solid and well-grounded. what's more interesting is the cost: ~$0.26/h on L4 spot. single LLM. no vector database, no embedding model, no workflow/pipeline engineering.
using kv cache as document store is nothing new, like the old CAG paper. but with quantized kv cache and modern attention (hybrid SSM-attention, GQA, MQA, MLA), the economics are changing fast. if we solve cold-prefill speed and decoding speed, and budget GPU costs keep dropping, the future of search could be vectorless. radical, but possible.
Logan Yang retweeted
Software horror: litellm PyPI supply chain attack.
Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.
LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.
Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.
Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.
Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.
Logan Yang retweeted
Dan says he's got Qwen 3.5 397B-A17B - a 209GB on disk MoE model - running on an M3 Mac at ~5.7 tokens per second using only 5.5 GB of active memory (!) by quantizing and then streaming weights from SSD (at ~17GB/s), since MoE models only use a small subset of their weights for each token
Logan Yang retweeted
The only to-dos method that has worked for me:
Every week I make a weekly note, and write my to-dos for the week. I may add more items to it during the week.
If any items didn't get done I roll them over to the next weekly note or drop them.
That's it.
Logan Yang retweeted
Pure MLX of UMAP, t-SNE, PaCMAP, TriMap, DREAMS, and CNE for Apple Silicon. Beyond the algorithms, I also built a circle-splatting renderer in MLX - scatter-add alpha blending on Metal, piped to H264 hardware encoding. So 70K points, 784 dimensions, from raw data to rendered video < 5s on M3 Ultra.
pip install mlx-vis
Paper: arxiv.org/abs/2603.04035
Code: github.com/hanxiao/mlx-vis
RT @william_cobb: I see a lot of people talking about how awesome Claude is with Obsidian, but you know what works even better? Obsidian Co…
Logan Yang retweeted
The paper says the best way to manage AI context is to treat everything like a file system.
Today, a model's knowledge sits in separate prompts, databases, tools, and logs, so context engineering pulls this into a coherent system.
The paper proposes an agentic file system where every memory, tool, external source, and human note appears as a file in a shared space.
A persistent context repository separates raw history, long term memory, and short lived scratchpads, so the model's prompt holds only the slice needed right now.
Every access and transformation is logged with timestamps and provenance, giving a trail for how information, tools, and human feedback shaped an answer.
Because large language models see only limited context each call and forget past ones, the architecture adds a constructor to shrink context, an updater to swap pieces, and an evaluator to check answers and update memory.
All of this is implemented in the AIGNE framework, where agents remember past conversations and call services like GitHub through the same file style interface, turning scattered prompts into a reusable context layer.
----
Paper Link – arxiv. org/abs/2512.05470
Paper Title: "Everything is Context: Agentic File System Abstraction for Context Engineering"
Logan Yang retweeted
Someone points me PaCMAP. 3-phase optimization makes for a much more dramatic convergence than UMAP. Both now pure MLX on Apple Silicon. Fashion-MNIST 70K in ~2s.
github.com/hanxiao/pacmap-ml…
github.com/hanxiao/umap-mlx
Logan Yang retweeted
OpenRouter weekly token consumption has exploded 12.7x in the last year to ~12.1 trillion tokens/week or 662T tokens/year run rate.
To give you a sense of this scale, Azure did 100T tok/quarter 6mos ago.
OpenRouter is responsible for about as much inference as ALL of Azure.
Logan Yang retweeted
Why @obsdmd is 100% user-supported and not backed by VC investors:
1. We want to stay small, we don't need to hire lots of people
2. We follow strict principles that we do not want compromise
3. Our users are happy to support us, we don't need VC money
Obsidian will not exist forever, no app will. However, the files you create in Obsidian are yours, and can hopefully last for generations. VCware is built with a five year horizon, it is not built to live on for decades.
Many startup founders raise VC money because they need the upfront capital to build their product, or they see it as a shortcut to growth. For some products the capital truly is necessary, but too often it's fueled by impatience and the inertia of Silicon Valley.
In the short term, VCware tends to subsidize pricing to acquire users. It's easier to grow if your product is cheap or free. But this generally comes at the cost of hoarding user data, and locking in customers. Once you're in you can't get out.
To keep raising money, VCware must paint an increasingly enormous vision of their future, which becomes impossible to live up to. This leads to increasingly disparate priorities that gradually make the product worse. What starts off as a useful app become burdened with crap.
Eventually all VCware must exit. That means being acquired or going public to pay back investors. It's expected that 9 out 10 startups will fail. That's just part of the math in a VC portfolio. The startups that have big exits pay for the ones that fail.
It is now possible for tiny teams to make principled software that millions of people use, unburdened by investors. Principled apps that put people in control of their data, their privacy, their wellbeing. These principles can be irrevocably built into the architecture of the app.
Principled people have always been able to make principled software. The difference is that now you need far less money and far fewer employees to reach far more customers. That wave is only just beginning.
If you have principles and enough patience, being 100% user-supported is by far the most fun way to build.
ALT Why Obsidian is 100% user-supported and not backed by VC investors: 1. We want to stay small, we don't need to hire lots of people 2. We follow strict principles that we do not want compromise 3. Our users are happy to support us, we don't need VC money Obsidian will not exist forever, no app will. However, the files you create in Obsidian are yours, and can hopefully last for generations. VCware is built with a five year horizon, it is not built to live on for decades. Many startup founders raise VC money because they need the upfront capital to build their product, or they see it as a shortcut to growth. For some products the capital truly is necessary, but too often it's fueled by impatience and the inertia of Silicon Valley. In the short term, VCware tends to subsidize pricing to acquire users. It's easier to grow if your product is cheap or free. But this generally comes at the cost of hoarding user data, and locking in customers. Once you're in you can't get out.
ALT To keep raising money, VCware must paint an increasingly enormous vision of their future, which becomes impossible to live up to. This leads to increasingly disparate priorities that gradually make the product worse. What starts off as a useful app become burdened with crap. Eventually all VCware must exit. That means being acquired or going public to pay back investors. It's expected that 9 out 10 startups will fail. That's just part of the math in a VC portfolio. The startups that have big exits pay for the ones that fail. It is now possible for tiny teams to make principled software that millions of people use, unburdened by investors. Principled apps that put people in control of their data, their privacy, their wellbeing. These principles can be irrevocably built into the architecture of the app. Principled people have always been able to make principled software. The difference is that now you need far less money and far fewer employees to reach far more customers.
Logan Yang retweeted
I am prone to existential depression... My mind easily drifts towards thinking that in the grand scheme of things nothing we do really matters, so why bother doing anything at all?
But within this fear is a desire for some kind of external validation. I like this quote by Kubrick: "The most terrifying fact about the universe is not that it is hostile but that it is indifferent. However vast the darkness, we must supply our own light."
It helps me reframe life from being outside-in to inside-out. Inside of you is a tiny candle and you are in control of it. Keep it alive and continue to expand that light. For yourself. That's all.
Yes, nothing matters. But also, nothing matters! You are free. You are free to take that inner candle anywhere you want. Its glow will naturally grow the things around you.
xitter.cf/kepano/status/10590998…
