anyone else received a critical security alert from @adcock_brett’s @hark_labs ai signing into their google account unprompted?
make sure to opt out of data use by AWS AI services for every region at settings.aws.com/projects
Jacob retweeted
[continued]
I say "almost" because there is going to be one way to maintain secrecy. Software as a service with the executable hiding behind a network connection to the cloud. But as of now, we must assume that anything shipped as an executable, or even a firmware image, is as transparent as glass. It won't keep its secrets for any longer than it takes for somebody to be motivated to throw an LLM at it.
I did not see this coming. When I wrote down the theory of open source, 30 years ago now, I thought closed source would be gradually driven towards extinction by cost gradients, but survive indefinitely in certain niches. I did not foresee it being wiped out in a technoapocalypse.
Ironically, I thought one of the application areas in which closed source would persist longest was games.
There are still some obstacles. In US law, decompilation of a binary is a derivative work of the binary that falls under whatever copyright it had. However, it is also settled law that if you decompile binary code to a precise specification of what it does, then generate fresh code from that without looking at the decompiled stuff, you're in the clear. This was the case law that allowed PC clones to exist, after Phoenix Technologies reverse engineered the original IBM PC BIOS.
The two-step process - binary to specification to unencumbered code - will no longer takes a large number of programmers and years of development time. With an LLM, it's now a thing you can do in a day. Ubiquity will make it impractical to prosecute all the people who might skip the specification step.
Source code can also be covered by patents. You can be able to see their source code for a patented technique and not be able to use it without violating the law. Linux has evolved practices for dealing with this problem - one is not shipping patented video codecs, but requiring you to download them as plugins from jurisdictions where U.S. patents can't be enforced. This presents patent holders with the impractical challenge of individually suing millions of end users, assuming it can even figure out who they are. To date, AFAIK, this has not been attempted.
And that's about it. If there are any other ways left to retain software secrecy and lock-in than SaaS or patents, I can't think of any. Well, you could epoxy-pot a firmware ROM, I suppose, but that can be defeated with a heat gun and a dental pick. Device manufacturers will learn not to pay for an assembly step that has become useless.
Forced migration from shipped binaries to tied cloud services will be tried - Adobe pioneered this, and Microsoft is pushing it as hard as they can given that their OS is a binary that has to run locally. Both companies are seeing massive user revolts over this. There is good reason to doubt that it's a strategy that can hold customers in the long term.
Also, a lot of things can't safely be tied to the cloud at all, because they can't tolerate a random network outage. Machine tools, medical devices...
A whole lot of proprietary software business models are going to collapse. Hard. One that I think might survive is tax software; being able to decompile it doesn't necessarily do you a lot of good because its actual value is tracking a ruleset that changes over time and has to be maintained by vendor specialists. But cases like this are unusual.
I think some dirty laundry is going to get aired, too. It has long been rumored that the reason graphics card manufacturers are so stubborn in their secrecy is that they've all been committing massive intellectual-property theft on each other for decades. If this is true, it will be exposed, and the lawsuits will be entertaining.
We're entering a new world, with a lot of ancient comfortable assumptions being blown up. It's going to be fun to watch.
2/2
will claude keep giving me these free resets after they’re accountable to wall street
43%no, 1-time pumping
29%no, not in long run
0%yes, to compete w codex
29%yes, if ur data is usefl
14 votes • Final resultsinstead of just deciding, imagine an entire toolkit of neurosymbolic operators 🤔
learning structured differentiable programs🤔
hope @tensacode 2022 inspired something
github.com/TensaCo/tensacode…
hype-free explanation of jev:
jev does not replace gpt / claude
jev is just a *really* smart switch statement
like if 2016 ml classifiers got 2026 levels of intelligence
it's a new* type of tool that will make a lot of workloads insanely fast, cheap, and accurate
* = and by new, i mean rebranded
~~~
it needs a predefined set of options and it will tell you which one to take
it cannot:
- write code
- generate natural language
- reason step by step / show its work
- produce any output you didn't define in advance
- pick from more than ~255 options in one shot
but it can:
- classify, route, score, rank
- give confidence
- pick the right branch, tool, model, or sub-agent
- judge / verify / guardrail an llm's output
- label tons and tons of rows
~~~
i'd imagine a lot of workflows that look like:
llm proposes options → jev decides → code executes
and i see this fitting *really* well with code mode and mcp
~~~
implying this will lead to agi seems incredibly far fetched to me, but i don't want to discount the types of applications that this will make possible
Jacob retweeted
Introducing a complete map of the male fruit fly brain and central nervous system, created in partnership with @HHMIJanelia. With over 166,000 neurons, this is the largest cellular brain map to date. Learn more: goo.gle/4qQTtIW
Jacob retweeted
iiiiiiIIIT'S TIIIIME
we proudly present:
solidSF v1.0
hardware engineers, this one is for you
a platform of platforms, amplifying or supporting all of the tools you love and hate
folded into a clean mesh of cohesive hardware engineering excellence
so what can it do?
Jacob retweeted
Introducing Prime Agent:
A self-improving RLM harness for coding and long-running autonomous tasks.
Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.
6/ and every region allocation and connecting edge is a decision you can read back. these are three views of ONE declared ward connectivity, each highlighting a different link: (nurse)→patient across frames, (state)→risk, state persisting over time, with the causal reason it exists. you're not stuck probing and reverse-engineering what the model learned because you wrote the graph
that's the interpretability a declarative schema brings: the latent tensor is POINTABLE. like memory mapping for compilers but on the latent diffusion canvas: region bounds are struct offsets, the topology is a calling convention, a serialized schema is an ABI. two models sharing a schema can exchange latent state directly — no tokenization, no re-encoding. all yawn-able when the diffusion is implemented in torch on a GPU but very serious when we have to repurpose cat-video-gen optimized silicon for real work
ALT nurse → patient (across frames): a nurse's actions change patient physiology next step
ALT state → risk: risk may only read from real physiological state
ALT state(t-1) → state(t): physiology is continuous, so persistence is wired in
ALT the serialized schema ("the ABI")
7/ (longpost) and this points us closer to the one PoC we have for general intelligence: the human brain. many neuroscientists have long argued that the cortex's functional connectivity is a reflection of the causal structure of its environment.
large scale functional connectivity (macro-wiring) is largely specified by the connectome, fixed. wheras the micro-weights are learned by synaptic plasticity over a lifetime. fixed topology + learned weights. that's the split canvas engineering makes
and the cortical connectivity matrix from post 2 in this thread? it's the SAME object as a canvas attention mask: source × destination, block-diagonal within a network, sparse specific cross-network edges, and every cell its own an operator type. the connectome IS a canvas topology
to test this, I wired 23 Destrieux regions over 42 known cortical pathways (mostly within the default-mode loop, ventral visual stream, and A1→Wernicke→Broca, prefrontal→premotor→motor) and trained it on real TRIBE v2 cortical predictions. the canvas engineering approach saturates R²=0.825 on single-timestep dynamics which matches a fully-dense model at 19.6% of the connections and 5× faster!
pause and chew for a second if this isn't clicking because I feel like we are very silo'ed in ML and could save a lot of time by learning from neurosci. (tho also please recognize that biology has never been as simple as scientists understand it to be; i'm just drawing a connection i haven't seen anyone notice yet). i will be unpacking this all in the future
ALT you declare the connectome; SGD tunes the synapses. not all regions shown
ALT the cortical connectivity matrix = a canvas attention mask
ALT predicts real TRIBE v2 cortical dynamics
ALT the canvas regions ARE cortical areas (on a real brain)
paper: commandagi.com/research/canv…
code: github.com/commandAGI/canvas…
docs + runnable examples (cartpole, vehicle fleets, hospital model, brain connectivity, more): commandagi.github.io/canvas-…
$ pip install canvas-engineering
8/8
5/ ok so that was the main idea, but there's a higher layer of abstraction: rather than allocating latent territory on the canvas by hand, the canvas-engineering library serves as a compiler for typed schema declarations which allows declaring complex relationships explicitly and transparently in code. you write the causal structure as a python object and canvas-engineering compiles that graph for you into regions and attention masks
in this example I defined some of the large-scale structure within a hospital ICU ward with 6 patients, organ-level physiology, 4 nurses with fatigue dynamics, insurance/staffing pressure, and families. The canvas engineering compile_schema() call allocates and packs 199 regions and 1,077 connections on the diffusion canvas, heart_rate updates every frame, creatinine every 24, etc., etc. See the figure below for the 1:1:1 correlation between the schema, the casual structure, and the canvas allocation:
ALT schema → the causal graph it denotes → 199 packed ICU regions
3/ the split is: you fix whether an edge exists, its direction, its temporal extent. gradients (data or intrinsic losses) determine what flows along it and everything inside regions
symbolic is macro. neural stays micro. and there's no interface between them, because the symbolic layer literally IS the attention mask. if there's no path between two regions, their independence is exact by construction, not regularization
ALT the four core equations
4/ with this framework, a standard markovian diffusion policy is just a two-node canvas where (observation) → action and (observation + action) → observation.
multi-agent is more complex but still essentially each agent self-attending over its own obs and actions and coordinating only through declared cross-edges
ALT diffusion policy = the two-node canvas, across frames
ALT multi-agent = the same primitive composed
ALT real ↔ neural: 4 robots in a field, the CanvasLayout+CanvasTopology, and the canvas the compiler builds
2/ how it works: you declare the latent block regions on the canvas, their connectivity, their temporal update frequencies, and their loss roles. a compiler converts all of that into attention masks, loss weights, and frame mappings on a stock pretrained diffusion transformer
but instead of uniform reverse diffusion over a flat bag of tokens, the connectivity you declared means only specific blocks are even allowed to influence each other (attention, not conv). that hard constraint induces a causal graph inside the reverse diffusion dynamics. an interaction graph you define explicitly (3rd image)
ALT declarations → compiled canvas + overlaid connection arrows
ALT the five time slices of the same layout
ALT the topology compiled to an attention mask
but what if you could map the causal structure of any problem domain onto that neural geometry? repurposing all that silicon optimized for generating pretzels onto real problem domains? today i want to share a new approach i've been exploring:
CANVAS ENGINEERING: PROMPT ENGINEERING FOR REVERSE DIFFUSION LATENT SPACE DYNAMICS
(1/8) 🧵
Jacob retweeted
right monitor is 20 codex instances. left monitor has situational awareness on autoscroll. center monitor is my word doc mainfesto. two keyboards, one for both hands. left airpod is dwarkesh x eric jang, 3x speed. right airpod tchaikovsky. meta quest 3 overlays my HUD: heart rate, words per minute, blood caffeine content. one assistant hooks me to an iv of chinese peptides, cocktail. the other feeds me kimchi. my unitree robot steps in when my posture slouches. blue light beams down on me in my herman miller chair. efficiency. no wasted movement. no wasted thoughts. think you can keep up with me? good luck. this is just for my morning emails.