A C/C++ based package for advanced data transformation and statisical computing in R. Account managed by the author. #rcollapse
CRAN and Github
Joined June 2020
- Tweets197
- Following29
- Followers947
- Likes84
Pinned Tweet
The article on "collapse: Advanced and Fast Statistical Computing and Data Transformation in R" has now been published (open-access) in the Journal of Statistical Software: jstatsoft.org/article/view/v…
It offers a concise yet thorough introduction to the package. #rstats #DataScience
I have created {collapse} and {fastverse} skills for claude code, as well as an experimental skill for scientific plotting in R: github.com/fastverse/skills
Enables agents to write parsimonious and super-fast production-grade R code that is unlikely to break.
#rstats #DataScience
Recording of my talk on {collapse} and the {fastverse} at the Bank of Portugal‘s workshop on „Speeding up Empirical Research: Tools and Techniques for Fast Computing“ in December is now online: youtu.be/qO5dHIPsfK8?si=1-mY… #Rstats #DataScience
Updated windows benchmarks for in-memory database-like operations by Adrian Antico show that {collapse} still leads on lagging and casting benchmarks (not covered in DuckDB benchmarks) and remains overall very competitive: github.com/AdrianAntico/Benc… #Rstats #DataScience
I've released a new package {flownet} for efficient transport modeling and graph manipulation: sebkrantz.github.io/flownet/. It builds on 7 {fastverse} libraries, most notably {collapse}, from which it imports 60 functions. Thus, another great learning resource for developers #rstats
I'm excited to share the release and rOpenSci publication of dfms 1.0 (docs.ropensci.org/dfms), a high-performance, {collapse} and {RcppArmadillo}-based, and feature-rich, implementation of Dynamics Factor Models in R. More in the release blog post at: sebkrantz.github.io/Rblog/20…
A few updates:
(1) new fastverse domain at fastverse.org
(2) the collapse (kit) repos moved to github.com/fastverse/collaps… (/kit) and site to fastverse.org/collapse (/kit)
(3) A group of maintainers has been given access
(4) collapse has a DeepWiki deepwiki.com/fastverse/colla…
{collapse} 2.1.0 is out! It introduces a new fslice() function (sebkrantz.github.io/collapse…), a new theory-consistent weighted quantile algorithm (sebkrantz.github.io/collapse…) with excellent properties. And some convenience features such as join requirements: #rstats #DataScience
The {collapse} arXiv paper has just been updated - following extensive revision: arxiv.org/abs/2403.05038. I believe it is a great resource for anyone doing scientific computing with #rstats.
There is now a #fastverse benchmark wiki (github.com/fastverse/fastver…) where users can freely contribute benchmarks. If you have benchmarks involving {fastverse} packages ({collapse}, {data.table}, etc., including extensions) please contribute them (takes 1 min) #rstats #DataScience
It's nice to see an increasing number of #rstats packages using {collapse}. A developer focused vignette was long planned and now it is here - with modest advice on writing efficient R package code in general and using {collapse} in particular: sebkrantz.github.io/collapse…
{collapse} is now also on BlueSky (bsky.app/profile/rcollapse.b…) and I am also there (bsky.app/profile/sebkrantz.b…) [and on Mastodon: fosstodon.org/@sebkrantz]. I will repost {collapse} posts but also share about research/data. This X account will remain active. #rstats
collapse retweeted
Check out the latest package to be granted the Seal of Approval: {collapse} by Sebastian Krantz!
{collapse} is a partner package, that implements various data transformation and statistical analysis tasks using ultra fast C/C++ implementations.
rdatatable-community.github.…
New independent benchmark by Adrian Antico: github.com/AdrianAntico/Benc…
Setup:
- large local Windows machine
- real data
- broad range of tasks
- scripts executed inside Rstudio and VScode
-> shows that {collapse} is an absolute top performer in this setting #rstats #DataScience
{collapse} v2.0.15, already available via install.packages("collapse", repos = "fastverse.r-universe.dev"), adds wide/recast pivot()'s with aggregation, including some hard-coded internal functions. A game changer for pivot tables in R. More at sebkrantz.github.io/collapse…. #rstats
An article on {collapse} is available on arXiv: arxiv.org/abs/2403.05038 (submitted to Journal of Statistical Software). It highlights the aims and added value of collapse and its cutting-edge performance for many complex statistical tasks in #rstats. Please consider sharing it.
{collapse} has been benchmarked in the DuckDB benchmark: duckdblabs.github.io/db-benc…, and is pretty competitive on 0.5-5Gb (laptop-grade) operations. A surprise is that it seems to be the only framework next to DuckDB to be able perform large data joins (50Gb) efficiently. #rstats
I’m thrilled to announce the release of {collapse} 2.0, adding blazing fast joins, pivots, a flexible namespace, and many other features. It is a remarkable piece of R software and capable of enhancing the workflow of all R users. Spread the word #rstats
sebkrantz.github.io/Rblog/20…