The most advanced data processing framework for #PHP by @norbert_tech

Joined January 2025
Something BIG is coming to Flow #PHP with next release 💪
1
3
154
Next release @flowphp is going to be fully schema driven! Historically extractors could yield different schema per batch. Now all extractors are going to be self descriptive, meaning they always yields rows with the same shape. Thanks to this we can know the output schema 👇
1
1
1
103
If he would go with @flowphp there would be no outage, no memory abuse, just clean, light, stable stream of data moving it from oneplace to another. We also do support KeySet pagination style natively flow-php.com/work-shop/bluep… we even have a blueprint that has it all in one place
A job failed in production at 2am. Memory limit exhausted. Process killed. The Artisan command exported orders. It worked fine in development with 500 rows. In production with 800,000 rows, it consumed all available PHP memory and died. The fix everyone reaches for: increase memory_limit. That's the wrong answer. The right answer is understanding which of Laravel's four large-dataset tools is correct for the specific operation. Here's what I learned from measuring all four on 1,000,000 rows. Order::all() with 100,000 rows used 284MB of memory. Extrapolated to 1,000,000 rows: 2.84GB. The default PHP memory limit is 256MB. The math ends in a fatal error before the loop even starts. cursor() used 8MB on 1,000,000 rows. The same 8MB. Constant. It holds the database connection open and yields one Eloquent model at a time. Only one object in memory at any moment. The cursor() gotcha I hit immediately: you can't run other queries on the same connection while the cursor is open. Customer::find() inside the foreach loop threw PDOException: Cannot execute queries while other unbuffered queries are active. The fix: eager load with() before the loop, not inside it. The chunk() bug I hit later — the dangerous one. I was processing a column I was also filtering on. Chunk 1 queries WHERE processed = false LIMIT 100 OFFSET 0, updates those 100 rows to processed = true. Chunk 2 queries WHERE processed = false LIMIT 100 OFFSET 100. But those 100 rows are now gone from the result set. OFFSET 100 skips the next 100 unprocessed rows. The command completed without errors. About half the rows were never processed. chunkById() fixed it. ID-based pagination doesn't depend on the result set order. Each chunk queries WHERE processed = false AND id > {lastId}. Modifying 'processed' doesn't affect which rows appear in the next chunk. The numbers from the measurement: cursor(): 8MB, constant regardless of row count lazy(): 9MB, with Collection methods chainable chunk(500): 12MB, safe for writes chunkById(500): 12MB, safe for writes AND for modifying the filtered columns The time differences are smaller than expected. Memory is the only dimension that matters at scale. The rule I use now: if I don't know the size of the result set, I don't use get(). Correct defaults for Artisan commands and background jobs: chunkById() for writes, lazy() for reads, get() only for queries with an explicit limit(). Full post with all gotchas, the complete chunk() skip bug explanation, and the memory-safe Artisan command template on the article. #Laravel #LaravelDeveloper #PHPDeveloper #PHPPerformance #DatabaseOptimization #BackendDevelopment #PHPFramework #SoftwareEngineering #LaravelTips #WebDevelopment #PHPCommunity #BackendEngineering #TechBlog #MySQL #EloquentORM #PHPDevelopment #ServerPerformance #Laravel13 #CodeOptimization #ApnaHive sadiqueali.medium.com/larave…
1
1
3
287
I'm very excited to announce the launch of #Flow PHP Work-Shop flow-php.com/work-shop The goal is to make some extra money to cover costs of building Flow. Flow has always been Open Source, and now you can help me keep it this way!
1
7
246
Flow #PHP just got much faster, it also received it's own dedicated binary, immutable, self descriptive and evolving data format, called Floe. You can read more about it at our latest blog post: flow-php.com/blog/2026-07-23…
2
11
303
Flow #PHP 0.41.0 is out! I managed to put together a quick blog post summarizing the most important changes / additions. Read more at: flow-php.com/blog/2026-06-29…
2
14
769
Working with #PHP, #PostgreSql and #Symfony? You might want to check flow-php.com/documentation/c… - query builder with full syntax support - migrations - define your schemas in php/yaml - SQL AST Parser - Client that supports static analysis types narrowing
3
1
39
2,432
Apache Arrow extension for #PHP is now available also as precompiled binaries. Just download the right one from: github.com/flow-php/arrow-ex… and enjoy parquet performance/efficiency unseen anywhere in PHP when it comes to files 💪
1
7
33
1,324
1/9 🧵 Parquet is an amazing file format, almost like an immutable database in a single file. One of the cool features is that each column can have different encoding and compression. Let's take a look at Delta Binary Packed encoding.
1
1
1
2
528
Version 0.15.0 is here! 🎉 Heads up: There's a BC break—schema-related classes have shifted to a new namespace. Dive into our upgrade docs for the scoop. Spotlight on this release: Exciting updates for Dbal and Google Sheets Adapters. Get the full rundown: github.com/flow-php/flow/rel…
1
4
333
Exciting changes ahead for the Flow #PHP Project! Starting today, get ready for fresh releases every two weeks 🚀 Of course, we'll swiftly tackle any critical bugs or security issues. Dive into the details here: flow-php.com/blog/2025-03-16…
1
1
1
5
396
When diving into datasets, defining a schema is your go-to move for reliability and consistency. Flow #PHP CLI app can effortlessly detect schemas from any compatible data sources. Today, let me introduce you to column statistics:
2
1
1
8
757
Hey, don’t do that! High coupling with the framework makes maintenance and upgrades a nightmare, and it also limits your system’s scalability. I know it’s a quick way to build something, and it might work for a while, especially if you’re building a CRUD app. But here’s the thing: a good framework should stay out of your way. Frameworks are meant to solve specific problems, not be the center of your system. Laravel and Symfony are great for: - UI (CLI/Http) - Routings - Backend HTML rendering - Handling and validating input/output And probably a few other things. Try to build your system’s core as if it knew nothing about the framework. That way, you’ll have a more flexible and maintainable system. That’s what sets software engineers apart from "framework developers". Frameworks come and go, like Zend. Don't let your system die with the framework (or framework version)
Benjamin Crozat
@benjamincrozat5 Feb 2025
Laravel only needs one thing from you. That you embrace it. Kiss it even. Delegate as much work to the framework as possible. Don't reinvent the wheel. For instance, are you feeling like creating a src directory? Don't. The app directory is what you're looking for. ✓ Feeling like creating modules? Don't. Subfolders inside the default structure is what you're looking for. Keep it familiar for your coworkers to ensure maximum productivity. ✓ Feeling like coding a feature from scratch? Don't. Laravel probably offers it already. And third party packages are already there too. ✓ Be the kind of developer who delivers and leaves a clean legacy.
1
5
309
Nice one! I'm glad to see posts about data processing in #php For those who want to take data import to the next level, scaling up to dozens of millions, flow-php.com is a solution. It supports all popular data formats, including CSV, JSON, DB, Elasticsearch, Parquet, and XML. It can also be easily integrated with Salon PHP to bring data from external APIs. This allows for on-the-fly data transformations with powerful schema evolution with minimal resource consumption.
Great read about importing a huge amount of data 👍 Thanks @JonPurvis_ jonathanpurvis.co.uk/importi…
3
11
1,641