Asking an AI to “critique my plan” can leave you with more objections and no clearer idea of what to do next. Two skills are worth looking at, depending on where you’re stuck. If the plan is still vague, Matt Pocock’s grill-me helps you work through it. The agent asks questions like “Who is this for?” and “Which features actually matter?”, then follows up on your answers until the decisions are clearer. If you already have a plan but want to challenge it before committing, paperthin’s /hate asks for the assumption that could undermine the whole thing and the cheapest way to test it. That might mean checking whether anyone wants a product with a small demo before spending two months building it. More of the development workflow seems to be moving out of harness-specific features and into skills written in plain language. The harness still runs the tools, but the procedure can live in a file you can read, edit, and reuse. Choosing the right skill for the situation has become a basic part of how I work. Clarifying a plan and testing its assumptions need different procedures, even if both could start with “critique this.” I’ll link both below.

I placed 8th and 7th in two Kaggle competitions back to back, and I still don't know what the data in either of them looks like. I never opened it. One was an OCR problem, reading when and where museum specimens were collected from photos of old, handwritten labels. The other was a tabular competition with 3,540 teams. Completely different problems, and the same approach held up on both. That's the part I didn't expect. The setup itself is not complicated. I wrote down what I'd learned from 50+ ML competitions as markdown: how I order experiments, what I check after a failure, how I think about submissions. Then I let an agent run on top of that, choosing experiments, submitting, reading the score, and writing its own retro before the next attempt. It made 180+ submissions in the first competition alone. I still think sitting with the data matters. But a lot more of that intuition moved into documents and a loop than I would have guessed, and seeing it happen twice in a row is making me rethink how I approach these competitions.

데이터를 아예 안 보고 국제 머신러닝 대회 Kaggle 상위 0.4%를 달성했습니다. 3540팀이 참가한 Kaggle Playground Series S6E8에서 Public 7위, Private 13위로 마무리했습니다. 사실 어제도 최종 8위로 마감한 대회가 있는데, 제 방식이 다양한 대회에서 반복 가능하고, 재현 가능하다는 사실이 재밌네요. 대회는 전형적인 tabular ML 대회였기에 제가 참가한 50개 넘는 ML 대회에서 쌓은 실험 순서, 실패 회고, 제출 전략을 .md로 일반화해둔 게 주요했고요. 따로 데이터를 보면서 피처를 파거나 도메인 직관을 쌓는 방식은 아니었습니다. 저도 아직 데이터가 어떻게 생겨먹은지도 모릅니다. 대신 paperthin 레포 기반 /goal 루프가 최적의 전략을 찾아 제출하고, 점수를 보고, 회고하고, 다음 실험을 고르게 하는 방식입니다. 이번에는 새로 나온 Grokbot도 활용해 봤는데, 제출 로그와 점수 변화 및 다른 사람들의 공개 코드 수집, 실패 패턴 분석, 다음 실험 후보 정리에서 도움을 받았습니다. 사용해 보니 꽤 좋아요. 보조 연구원이라는 인상이었습니다. Kaggle이 점점 “혼자 Kaggle을 푸는 것”보다 작은 autonomous research team을 운영하는 느낌에 가까워지고 있다고 생각합니다. 사람이 데이터를 직접 보면서 얻는 직관은 여전히 중요하지만, 그 직관의 일부는 문서화된 방법론, 점수 기반 회고, persistent agent loop로 꽤 많이 이전될 수 있어 보입니다.