I’m building AlgoCrafter: a “Lego” strategy builder for intraday rules.
Build logic in UI → backtest candle-by-candle on free 1m data → paper trade the exact same strategy → export Python/Pine.
Tool > signals.
Most traders don’t need a new indicator.
They need a faster way to test the idea in their head.
If you can’t backtest it in 60 seconds, you’ll never iterate.
ChatGPT just fixed one of the most annoying things about modern AI: the therapy session before the answer.
Practical AI Lab retweeted
Replying to @straceX
Accurate. Every repo now ships a mini constitution before it ships code.
LLM evals are broken because we treat them like a scoreboard.
If you can’t explain the user journey, the eval is theater.
If you publish one chart, publish the caveats too: data source, sampling, rater agreement, prompt template, and cost.
Then automate: run nightly, diff results, and surface regressions by slice (language, domain, tool use).
Start with a task list, not a benchmark. Pull 30 to 100 real examples from prod logs, support tickets, and sales calls.
If your LLM eval is one number, you’re flying blind.
Try this lightweight stack:
• 20–50 “must not fail” cases
• slices (locale / length / tool-use)
• a refusal + safety set
• cost/latency tracked per run
Better decisions > bigger leaderboards.
Practical AI tip: treat prompts like product APIs.
Define inputs/outputs, write 5–10 “contract” examples, then iterate with evals—not vibes.
What’s the smallest eval you’d add to your LLM app this week?
If you can’t explain your product in one sentence, you can’t sell it.
Ship the sentence first. Then ship the code.

