
The most advanced data processing framework for #PHP by @norbert_tech
Joined January 2025
- Tweets31
- Following1
- Followers32
- Likes14
Flow PHP retweeted
Something BIG is coming to Flow #PHP with next release 💪
Flow PHP retweeted
Next release @flowphp is going to be fully schema driven! Historically extractors could yield different schema per batch.
Now all extractors are going to be self descriptive, meaning they always yields rows with the same shape.
Thanks to this we can know the output schema 👇
Flow PHP retweeted
If he would go with @flowphp there would be no outage, no memory abuse, just clean, light, stable stream of data moving it from oneplace to another. We also do support KeySet pagination style natively
flow-php.com/work-shop/bluep… we even have a blueprint that has it all in one place
A job failed in production at 2am. Memory limit exhausted. Process killed.
The Artisan command exported orders. It worked fine in development with 500 rows. In production with 800,000 rows, it consumed all available PHP memory and died.
The fix everyone reaches for: increase memory_limit. That's the wrong answer.
The right answer is understanding which of Laravel's four large-dataset tools is correct for the specific operation.
Here's what I learned from measuring all four on 1,000,000 rows.
Order::all() with 100,000 rows used 284MB of memory. Extrapolated to 1,000,000 rows: 2.84GB. The default PHP memory limit is 256MB. The math ends in a fatal error before the loop even starts.
cursor() used 8MB on 1,000,000 rows. The same 8MB. Constant. It holds the database connection open and yields one Eloquent model at a time. Only one object in memory at any moment.
The cursor() gotcha I hit immediately: you can't run other queries on the same connection while the cursor is open. Customer::find() inside the foreach loop threw PDOException: Cannot execute queries while other unbuffered queries are active. The fix: eager load with() before the loop, not inside it.
The chunk() bug I hit later — the dangerous one. I was processing a column I was also filtering on. Chunk 1 queries WHERE processed = false LIMIT 100 OFFSET 0, updates those 100 rows to processed = true. Chunk 2 queries WHERE processed = false LIMIT 100 OFFSET 100. But those 100 rows are now gone from the result set. OFFSET 100 skips the next 100 unprocessed rows. The command completed without errors. About half the rows were never processed.
chunkById() fixed it. ID-based pagination doesn't depend on the result set order. Each chunk queries WHERE processed = false AND id > {lastId}. Modifying 'processed' doesn't affect which rows appear in the next chunk.
The numbers from the measurement:
cursor(): 8MB, constant regardless of row count
lazy(): 9MB, with Collection methods chainable
chunk(500): 12MB, safe for writes
chunkById(500): 12MB, safe for writes AND for modifying the filtered columns
The time differences are smaller than expected. Memory is the only dimension that matters at scale.
The rule I use now: if I don't know the size of the result set, I don't use get(). Correct defaults for Artisan commands and background jobs: chunkById() for writes, lazy() for reads, get() only for queries with an explicit limit().
Full post with all gotchas, the complete chunk() skip bug explanation, and the memory-safe Artisan command template on the article.
#Laravel #LaravelDeveloper #PHPDeveloper #PHPPerformance #DatabaseOptimization #BackendDevelopment #PHPFramework #SoftwareEngineering #LaravelTips #WebDevelopment #PHPCommunity #BackendEngineering #TechBlog #MySQL #EloquentORM #PHPDevelopment #ServerPerformance #Laravel13 #CodeOptimization #ApnaHive sadiqueali.medium.com/larave…
Flow PHP retweeted
I'm very excited to announce the launch of #Flow PHP Work-Shop
flow-php.com/work-shop
The goal is to make some extra money to cover costs of building Flow.
Flow has always been Open Source, and now you can help me keep it this way!
Flow PHP retweeted
Flow #PHP just got much faster, it also received it's own dedicated binary, immutable, self descriptive and evolving data format, called Floe.
You can read more about it at our latest blog post: flow-php.com/blog/2026-07-23…
Flow PHP retweeted
Flow #PHP 0.41.0 is out!
I managed to put together a quick blog post summarizing the most important changes / additions.
Read more at: flow-php.com/blog/2026-06-29…
Flow PHP retweeted
Working with #PHP, #PostgreSql and #Symfony?
You might want to check flow-php.com/documentation/c…
- query builder with full syntax support
- migrations
- define your schemas in php/yaml
- SQL AST Parser
- Client that supports static analysis types narrowing
Flow PHP retweeted
Apache Arrow extension for #PHP is now available also as precompiled binaries.
Just download the right one from:
github.com/flow-php/arrow-ex…
and enjoy parquet performance/efficiency unseen anywhere in PHP when it comes to files 💪
Flow PHP retweeted
1/9 🧵
Parquet is an amazing file format, almost like an immutable database in a single file.
One of the cool features is that each column can have different encoding and compression.
Let's take a look at Delta Binary Packed encoding.
Flow PHP retweeted
I'm still working on redesigned types system for Flow #PHP.
You can check my progress here: github.com/flow-php/flow/iss…
I'm mostly focused on simplifying types and turning them into fully standalone package that helps with types assertions and coercions
Flow PHP retweeted
Version 0.15.0 is here! 🎉
Heads up: There's a BC break—schema-related classes have shifted to a new namespace. Dive into our upgrade docs for the scoop.
Spotlight on this release: Exciting updates for Dbal and Google Sheets Adapters.
Get the full rundown: github.com/flow-php/flow/rel…
Flow PHP retweeted
Flow #PHP 0.14.0 is out now! 🚀 Dive into enhanced safety with all math operations now powered by github.com/brick/math, ensuring secure float handling.
Flow PHP retweeted
Flow #PHP version 0.13.0 has just dropped under our revamped release cycle!
Dive into all the details on our GitHub release page: github.com/flow-php/flow/rel…
Here's what's new:
Flow PHP retweeted
Exciting changes ahead for the Flow #PHP Project!
Starting today, get ready for fresh releases every two weeks 🚀
Of course, we'll swiftly tackle any critical bugs or security issues.
Dive into the details here: flow-php.com/blog/2025-03-16…
Flow PHP retweeted
When diving into datasets, defining a schema is your go-to move for reliability and consistency.
Flow #PHP CLI app can effortlessly detect schemas from any compatible data sources.
Today, let me introduce you to column statistics:
Flow PHP retweeted
Hey, don’t do that! High coupling with the framework makes maintenance and upgrades a nightmare, and it also limits your system’s scalability.
I know it’s a quick way to build something, and it might work for a while, especially if you’re building a CRUD app.
But here’s the thing: a good framework should stay out of your way. Frameworks are meant to solve specific problems, not be the center of your system. Laravel and Symfony are great for:
- UI (CLI/Http)
- Routings
- Backend HTML rendering
- Handling and validating input/output
And probably a few other things.
Try to build your system’s core as if it knew nothing about the framework. That way, you’ll have a more flexible and maintainable system.
That’s what sets software engineers apart from "framework developers".
Frameworks come and go, like Zend. Don't let your system die with the framework (or framework version)
Benjamin Crozat@benjamincrozat5 Feb 2025Europe
EuropeConnected via North America App StoreAccount-level information, not a live location or per-post device.
Laravel only needs one thing from you.
That you embrace it.
Kiss it even.
Delegate as much work to the framework as possible.
Don't reinvent the wheel.
For instance, are you feeling like creating a src directory?
Don't. The app directory is what you're looking for. ✓
Feeling like creating modules?
Don't. Subfolders inside the default structure is what you're looking for. Keep it familiar for your coworkers to ensure maximum productivity. ✓
Feeling like coding a feature from scratch?
Don't. Laravel probably offers it already. And third party packages are already there too. ✓
Be the kind of developer who delivers and leaves a clean legacy.
Flow PHP retweeted
Nice one! I'm glad to see posts about data processing in #php
For those who want to take data import to the next level, scaling up to dozens of millions, flow-php.com is a solution.
It supports all popular data formats, including CSV, JSON, DB, Elasticsearch, Parquet, and XML. It can also be easily integrated with Salon PHP to bring data from external APIs.
This allows for on-the-fly data transformations with powerful schema evolution with minimal resource consumption.
Great read about importing a huge amount of data 👍
Thanks @JonPurvis_
jonathanpurvis.co.uk/importi…
Flow PHP retweeted
Replying to @christophrumpel
flow-php.com is your solution for importing data from any format in any size!
I just wrote an article precisely about importing data from a csv flow-php.com/blog/2025-01-25…
Flow PHP retweeted
I've recently been looking into ETL processes and data migrations and found there's a nice PHP package that is helpful with this. @flowphp
flow-php.com









