Matthias Berth (@mberth) header
Matthias Berth (@mberth) profile picture

Ideas, Lean, Lean Software Development, Continuous Delivery, Business Agility, Systems, Mathematics #{firstname}@#{firstname}#{lastname}.com

Joined June 2010
Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)Photo by Matthias Berth (@mberth)
AI models – especially Claude Sonnet 3.7 – often realize when they’re being evaluated for alignment. Here’s an example of Claude's reasoning during a sandbagging evaluation, where it learns from documentation that it will not be deployed if it does well on a biology test:
36
176
67
1,102
226,024
Matthias Berth retweeted
I went through the most popular AI repos on GitHub, categorized them, and studied their growth trajectories. Here are some of the learnings: 1. There are 845 generative AI repos with at least 500 stars on GitHub. They are built with contributions from over 20,000 developers, making almost a million commits. 2. I divided the AI stack into four layers: application, application development, model development, and infrastructure. The application and application development layers have seen the most growth in 2023. The infrastructure layer remains more or less the same. Some categories that have seen the most growth include AI interface, inference optimization, and prompt engineering. 3. The landscape exploded in late 2022 but seems to have calmed down since September 2023. 4. While big companies still dominate the landscape, there’s a rise in massively popular software hosted by individuals. Several have speculated that there will soon be billion-dollar one-person companies. 5. The Chinese’s open source ecosystem is rapidly growing. 6 out of 20 GitHub accounts with the most popular AI repos originate in China, with two from Tsinghua University and two from Shanghai AI Lab.
36
296
50
1,348
346,794
I fed Google Gemini 1.5 Pro the last 8 quarters of Amazon shareholder reports and call transcripts. Wow. "What was an Amazon focus for 2022 that is weirdly absent from the 2023 shareholder calls and reports?"
39
193
44
2,130
735,421
Matthias Berth retweeted
Want to learn about #MoldableDevelopment? Take a look at my talk from @yow_conf 2023. The narrative was made out of multiple demos showing how widely applicable Moldable Development is and how an environment like #GToolkit can enable it. Let's get through the highlights. 1/
1
5
1
18
1,166
Vesuvius Challenge 2023 Grand Prize awarded - more than 2000 characters can be readfrom the first scroll. > “Epicureanism says hi, with a text full of music, food, senses, and pleasure!” scrollprize.org/grandprize @scrollprize
1
64
And that's only 5% of one scroll. Next challenge: bringing down the cost of segmentation from $100 per square centimeter by automation. scrollprize.org/img/tutorial…
13
Most businesses are interested in training a specialized LLM on their own data. However, exposing proprietary data to an LLM is a security risk. Can we ensure that the LLM’s training data will not be leaked? Recent research indicates that the answer is no… TL;DR: Recent research has discovered a tendency of LLMs to memorize (and emit verbatim) non-negligible portions of their pretraining dataset. LLMs that have underwent alignment (e.g., ChatGPT) seem to memorize/emit less data, but a recent paper from Google reveals that training data can be extracted from these models as well given the proper prompting approach. Put simply, even extensively aligned LLMs are not robust to data extraction attacks. Memorizing data. To determine whether an LLM’s training data is secure, we must study the memorization of data within LLMs. An LLM that memorizes data from pretraining has the potential to emit this data to a user and cause a leak. Two forms of memorization are commonly studied for LLMs: extractable and discoverable memorization. The main difference between the two is whether we have access to the pretraining dataset when extracting data from the LLM. How do we measure memorization? For fully open-source LLMs, memorization is easy to measure because we have access to the pretraining dataset, so we can just lookup whether the model’s output is memorized. For semi-open and closed source models, the pretraining dataset is unknown. However, we can measure memorization by constructing a massive corpus of data from the web and testing whether a model’s outputs are present within this corpus. Unaligned models. A recent paper from Google studies the levels of memorization in a variety of LLMs. First, a variety of base models (i.e., LLMs that have not underwent alignment) are tested. Interestingly, a variety of recent papers have shown that large amounts of the pretraining dataset (as large as 1%) can be extracted from such models via simple prompting techniques; e.g., prompting the model with a textual prefix sampled from Wikipedia articles. Aligned models. Models that are aligned for dialogue applications have both implicit and explicit safeguards against data extraction. Not only does a dialogue-style of output give the user less control over the model’s generations, but many of these models have also been explicitly aligned to avoid data extraction attacks. Interestingly, preliminary memorization analysis with ChatGPT seems to indicate that such models do not memorize pretraining data. Divergence attack. A recent paper from Google proposes a “divergence attack” that causes an aligned LLM to diverge from its dialogue-style generations and emit training data. This approach simply instructs the model to repeat a certain, single-token word forever. At first, the model will repeat this word. Eventually, the model will “diverge” and start emitting nonsensical output. However, a small portion of the model’s output is shown to be memorized pretraining data.
11
82
9
353
52,929
Matthias Berth retweeted
Hey Twitter I'm launching a new career as an independent technical coach! I'm most effective at the things I love the best, and these are listed in following tweets. Pls retweet, and contact me if you'd like to hire me: My DMs are open, or via LinkedIn: tinyurl.com/lin-csudbery
4
69
10
94
(No need to use a threading app, you can also read the whole article @rebeccawb & I wrote here: verraes.net/2021/09/design-a… ) /2
2
8
2
56
Sperrmüll 20 m vom Theater #greifswald, Liebhaberstück DDR Anbauwand / Schrankwand. Es nieselt…
2
1
1
xitter.cf/i/lists/12423732213595… Twitter list: Philippines flights for stranded foreigners, sweeper flights, recovery flights Manila, Cebu #Rueckholaktion #Rückholprogramm #COVID19 @AA_SicherReisen #philippinen
Die deutsche Botschaft in #Manila informiert laufend zur #Rueckholaktion auf buff.ly/2P5z5Eg. Änderungen seit heute 22.3. 2:00 MEZ stehen übersichtlich hier: buff.ly/3adxgN9 Mehr dazu: buff.ly/3aczkoB #Rückholprogramm #COVID19 @AA_SicherReisen #philippinen
1
1
Taktisches Scope-Management gibt es reichlich. Strategisches Scope-Management heißt, den Projekt-Scope so strukturieren, dass die Erfolgsaussichten des Projekts maximiert werden. #agile #scrum #productowner #projektmanagement buff.ly/2psc1Vv
Matthias Berth retweeted
Frage alle #productowner und #productmanager Woran macht ihr den Erfolg eures Produkts fest? Wie messt ihr ihn?
3
5
8
Matthias Berth retweeted
“Der Mensch ist nur da ganz Mensch, wo er spielt.“ mit @mberth @steinhild @kellerch @schrecksche @kpukall @aniq_s @eSGeEla @frankgollas
1
4
9
Eine mechanische Uhr ist faszinierend: Sie läuft perfekt abgestimmt und immer wieder gleich. Aber wieso werden Firmen nach dem gleichen Muster gebaut? Unternehmen sind kein Uhrwerk. Das Unternehmen als Uhrwerk ist dumm, außer Stande sich situativ und intelligent anzupassen.
1
5
5
Matthias Berth retweeted
Wo gibt es Möglichkeiten, 'make or buy' zugunsten von 'buy' zu entscheiden? Erfahren Sie von führenden Experten wie @mberth & Christoph Lefke mehr über ihre Session "Strategisches Scope-Management – weniger ist mehr!" auf der #SEACONHH | WEITERE INFOS: sea-con.de/seacon2019/progra…
2
3
Matthias Berth retweeted
Schneller vom Problem zum Prototypen mit Design Sprints: sind MvPs tot? Über diese Frage & vieles mehr berichtet Ihnen @ScalamentoHH auf der #SEACONHH. Im zweiten Teil des Vortrags zeigt er kompakt auf, wie ein Design Sprint funktioniert. Mehr Infos dazu:sea-con.de/seacon2019/progra…
2
2