Ricklee (@usricklee) profile picture
United States
Joined January 2016
Photo by Ricklee (@usricklee)
Ricklee retweeted
Calculus Notations ✍️
3
83
469
9,582
In LC circuit, energy has nowhere to go. So it oscillated.
7
96
3
556
29,800
ここから一次元 a を A = [a1 a2 ... ] の列空間に拡張します。
1
1
322
Ricklee retweeted
29
68
7
1,329
46,652
fantastic simple visualization of the self attention formula. this was one of the hardest things for me to deeply understand about LLMs. the formula seems easy. you can even memorize it fast. but to really get an intuition of what the Q,K,V represent and interact, that’s hard.
25
528
12
3,336
175,822
Build your own NotebookLM! A personal learning assistant that can: - process multi-modal data - scrape websites and YouTube videos - create a unified knowledge base - lets you do RAG over it - remember every conversation - generate a podcast 🎙️ 100% open-source.
I just built an open NotebookLM clone! Here's what it can do for you: - Process multi-modal data - Scrape websites and YouTube videos - Create a unified knowledge base - Lets you do RAG over it - Remember every conversation - Generate a podcast 🎙️ The idea here is not to reinvent the wheel but to understand how one of the most powerful tools for learning and research actually works, by building it step-by-step! So by the end of this video, you'll learn how to: ↳ Process multimodal data (text, audio, video, URLs, and YouTube videos) into a format ready for LLMs ↳ Store everything in a vector database for fast retrieval ↳ Add a memory layer that remembers conversations and preferences for a personalized experience ↳ Chat with your knowledge base or generate podcasts using a fully open-source, locally running text-to-speech model The podcast generation feature is my favorite part! There's something powerful about turning written content into conversational audio that you can listen to while doing something else. The entire code is 100% open-source. I've shared a link in the replies! ____ Don't forget to drop a like if you enjoy my videos. It shows me I should be making more content like this. Cheers! :)
14
83
3
512
51,738
Replying to @Qianaegean
刚理解错了以为是一个 GPU 可以部署多少个实例,如果是算用户请求的话,可以这样算: # 总显存需求 = 模型参数占用 + KV Cache占用 + 其他开销 # 模型参数占用 = 参数数量 × 数据类型字节数 KV Cache占用 = batch_size × seq_length × hidden_size × layers × 2 × 2 # 最大并发数 = (GPU显存容量 - 模型参数占用) / 单个请求的KV Cache占用 # 单个请求KV Cache = seq_length × hidden_size × layers × 2 × 2 bytes RTX 4090 (24GB),部署 Llama-7B INT4: 模型占用:3.5GB 可用显存:20.5GB 单个请求(512 tokens):约100MB (用户上下文长度、连续对话会显著增加 kv 占用) 理论并发数:~200个请求
1
2
90
if you’re confused about how large language models work this new animation from google deepmind should clear a lot of things up
171
312
110
3,289
874,259
User Mode to Kernel Mode
26
189
6,327
Things Every Developer Should Know: Concurrency is 𝐍𝐎𝐓 parallelism. In system design, it is important to understand the difference between concurrency and parallelism. As Rob Pyke(one of the creators of GoLang) stated:“ Concurrency is about 𝐝𝐞𝐚𝐥𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 lots of things at once. Parallelism is about 𝐝𝐨𝐢𝐧𝐠 lots of things at once." This distinction emphasizes that concurrency is more about the 𝐝𝐞𝐬𝐢𝐠𝐧 of a program, while parallelism is about the 𝐞𝐱𝐞𝐜𝐮𝐭𝐢𝐨𝐧. Concurrency is about dealing with multiple things at once. It involves structuring a program to handle multiple tasks simultaneously, where the tasks can start, run, and complete in overlapping time periods, but not necessarily at the same instant. Concurrency is about the composition of independently executing processes and describes a program's ability to manage multiple tasks by making progress on them without necessarily completing one before it starts another. Parallelism, on the other hand, refers to the simultaneous execution of multiple computations. It is the technique of running two or more tasks or computations at the same time, utilizing multiple processors or cores within a computer to perform several operations concurrently. Parallelism requires hardware with multiple processing units, and its primary goal is to increase the throughput and computational speed of a system. In practical terms, concurrency enables a program to remain responsive to input, perform background tasks, and handle multiple operations in a seemingly simultaneous manner, even on a single-core processor. It's particularly useful in I/O-bound and high-latency operations where programs need to wait for external events, such as file, network, or user interactions. Parallelism, with its ability to perform multiple operations at the same time, is crucial in CPU-bound tasks where computational speed and throughput are the bottlenecks. Applications that require heavy mathematical computations, data analysis, image processing, and real-time processing can significantly benefit from parallel execution. -- Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/bbg-social
5
175
1
849
59,183
Je me remets en condition pour flag du app - sys. Elle était pas mal cette cheat sheet sur les Buffer Overflow ! :P
1
2
18
Info sheet: binary representations of numeric values commonly used by hardware architectures (esp. x86) and programming languages. Unsigned integers, two's complement signed integers and IEEE 754 floating-point. #programming #coding #learning #cheatsheet #numbers #hardware
3
59
241
Ricklee retweeted
Preliminary block diagram for Zen 5 by AMD. Still awaiting on some details that AMD didn't clarify, such as the ITA BTB, Store & Load Queue sizes, etc. μarch Block Diagrams: bit.ly/32qLLew μarch Cheat Sheet: bit.ly/2JTplfJ #x86 #AMD #CPU
6
54
4
217
20,249
Kubernetes Native Sidecar Containers Explained 🔽 Kubernetes Pods support several types of containers: - init - regular - ephemeral - ...and since 1.28, "sidecars" Learn about the difference between them and when to use the new "native" sidecar type 👉 labs.iximiuz.com/tutorials/k…
1
42
305
12,424
How does Docker work? A comparison of Docker-based and non-Docker-based development is shown below. With Docker, we can develop, package, and run applications quickly: 🔹 The developers can write code locally and then build a Docker image and push it to a dev environment. In this way, the development work is shared within the team. The tests are run automatically in the dev environment. An image is a read-only template with instructions for creating a Docker container. We use a Dockerfile to define the steps to create the image and run it. 🔹 The above process can run incrementally when bugs are found or improvements are needed. 🔹 When dev testing is complete, the Docker image is pushed to the production environment (often on the cloud). Compared with traditional development without Docker, Docker is quite lightweight and fast, because only the changed part of Dockerfile is rebuilt every time we make a change. Over to you: Where are Docker images stored? – Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): bit.ly/3KCnWXq
4
146
6
771
54,188
btw these large scary math symbols are just for-loops
548
6,148
1,344
35,029
How Component Object Model (COM) works. #ThreatHunting #DFIR #Windows #Microsoft
4
161
4
858
82,074
How To Pick The Right Docker Image For Your Node.js App 👇 - Use an LTS release - Avoid bloated images - Balance simplicity/security All mainstream image candidates with the usage advice on a single diagram:
8
40
1
193
12,932
I've just discovered this amazing document showing super clearly the relation between the opcode and the instruction 🤯 pnx.tf/files/x86_opcode_stru…
15
497
31
2,026
Bayes’ theorem, clearly explained:
75
1,209
82
8,365
746,153