The most advanced data processing framework for #PHP by @norbert_tech

Joined January 2025
Something BIG is coming to Flow #PHP with next release 💪
1
3
154
Next release @flowphp is going to be fully schema driven! Historically extractors could yield different schema per batch. Now all extractors are going to be self descriptive, meaning they always yields rows with the same shape. Thanks to this we can know the output schema 👇
1
1
1
103
If he would go with @flowphp there would be no outage, no memory abuse, just clean, light, stable stream of data moving it from oneplace to another. We also do support KeySet pagination style natively flow-php.com/work-shop/bluep… we even have a blueprint that has it all in one place
A job failed in production at 2am. Memory limit exhausted. Process killed. The Artisan command exported orders. It worked fine in development with 500 rows. In production with 800,000 rows, it consumed all available PHP memory and died. The fix everyone reaches for: increase memory_limit. That's the wrong answer. The right answer is understanding which of Laravel's four large-dataset tools is correct for the specific operation. Here's what I learned from measuring all four on 1,000,000 rows. Order::all() with 100,000 rows used 284MB of memory. Extrapolated to 1,000,000 rows: 2.84GB. The default PHP memory limit is 256MB. The math ends in a fatal error before the loop even starts. cursor() used 8MB on 1,000,000 rows. The same 8MB. Constant. It holds the database connection open and yields one Eloquent model at a time. Only one object in memory at any moment. The cursor() gotcha I hit immediately: you can't run other queries on the same connection while the cursor is open. Customer::find() inside the foreach loop threw PDOException: Cannot execute queries while other unbuffered queries are active. The fix: eager load with() before the loop, not inside it. The chunk() bug I hit later — the dangerous one. I was processing a column I was also filtering on. Chunk 1 queries WHERE processed = false LIMIT 100 OFFSET 0, updates those 100 rows to processed = true. Chunk 2 queries WHERE processed = false LIMIT 100 OFFSET 100. But those 100 rows are now gone from the result set. OFFSET 100 skips the next 100 unprocessed rows. The command completed without errors. About half the rows were never processed. chunkById() fixed it. ID-based pagination doesn't depend on the result set order. Each chunk queries WHERE processed = false AND id > {lastId}. Modifying 'processed' doesn't affect which rows appear in the next chunk. The numbers from the measurement: cursor(): 8MB, constant regardless of row count lazy(): 9MB, with Collection methods chainable chunk(500): 12MB, safe for writes chunkById(500): 12MB, safe for writes AND for modifying the filtered columns The time differences are smaller than expected. Memory is the only dimension that matters at scale. The rule I use now: if I don't know the size of the result set, I don't use get(). Correct defaults for Artisan commands and background jobs: chunkById() for writes, lazy() for reads, get() only for queries with an explicit limit(). Full post with all gotchas, the complete chunk() skip bug explanation, and the memory-safe Artisan command template on the article. #Laravel #LaravelDeveloper #PHPDeveloper #PHPPerformance #DatabaseOptimization #BackendDevelopment #PHPFramework #SoftwareEngineering #LaravelTips #WebDevelopment #PHPCommunity #BackendEngineering #TechBlog #MySQL #EloquentORM #PHPDevelopment #ServerPerformance #Laravel13 #CodeOptimization #ApnaHive sadiqueali.medium.com/larave…
1
1
3
287
I'm very excited to announce the launch of #Flow PHP Work-Shop flow-php.com/work-shop The goal is to make some extra money to cover costs of building Flow. Flow has always been Open Source, and now you can help me keep it this way!
1
7
246
Flow #PHP just got much faster, it also received it's own dedicated binary, immutable, self descriptive and evolving data format, called Floe. You can read more about it at our latest blog post: flow-php.com/blog/2026-07-23…
2
11
303
Flow #PHP 0.41.0 is out! I managed to put together a quick blog post summarizing the most important changes / additions. Read more at: flow-php.com/blog/2026-06-29…
2
14
768
DX was never better! 🤩
I migrated entire Flow #PHP into Mago Linter / Formatter / Analyzer. What before was taking more than a minute now takes 5 seconds. One tool replaced 3 providing insane performance boost. Since Mago is also very strict it helped me to improve Flow code quality 💪
20
Working with #PHP, #PostgreSql and #Symfony? You might want to check flow-php.com/documentation/c… - query builder with full syntax support - migrations - define your schemas in php/yaml - SQL AST Parser - Client that supports static analysis types narrowing
3
1
39
2,432
Apache Arrow extension for #PHP is now available also as precompiled binaries. Just download the right one from: github.com/flow-php/arrow-ex… and enjoy parquet performance/efficiency unseen anywhere in PHP when it comes to files 💪
1
7
33
1,324
Arrow C++ #PHP Parquet extensio is going to be a performance game changer 🚀
flow-php/parquet got another significant performance boost 🚀 The next step is to create an extension for apache arrow cpp and make parquet in #php even faster!
1
39
We also moved all our examples to playground! Now each of them can be executed directly in the browser :) Layout of documentation also got slightly better
37
Because PHP can into data 💪
I wrote a new blog post about how to approach preparing data for analysis in distributed transactional systems. norbert.tech/blog/2025-08-12… Available in: 🇵🇱🇺🇸🇩🇪🇫🇷🇪🇸
1
3
407
Join and learn how to process data like a pro! 💪
Big data, small data, every data #php
98
1/9 🧵 Parquet is an amazing file format, almost like an immutable database in a single file. One of the cool features is that each column can have different encoding and compression. Let's take a look at Delta Binary Packed encoding.
1
1
1
2
528
Version 0.15.0 is here! 🎉 Heads up: There's a BC break—schema-related classes have shifted to a new namespace. Dive into our upgrade docs for the scoop. Spotlight on this release: Exciting updates for Dbal and Google Sheets Adapters. Get the full rundown: github.com/flow-php/flow/rel…
1
4
333
Processing large database based datasets is going to be much faster with this key set extractor 💪
Diving into Flow #PHP with a fresh Doctrine DBAL Extractor! Say goodbye to sluggish limit/offset and hello to speedy key-set pagination. 🚀 Mark your calendars for Monday’s release! Check it out: github.com/flow-php/flow/pul…
129