Data exploration and visualization with Python, pandas, seaborn, numpy and matplotlib

Joined October 2020
Include

Only show posts containing:

Exclude

Hide posts containing:

Time range
-
Minimum likes
I have a soft spot for tools that allow one to stay on the command line as much as possible. In that spirit, colorpedia buff.ly/39vZOU9 allows you to look up colours and palettes - the palette output I think is particularly cool.
One of the most important abilities of experienced programmers is a willingness to reshape the data to suit the question at hand. Here's an article I just wrote with a specific pandas example buff.ly/3rMtrXz
1
A round up of a few GUIs that work with pandas buff.ly/3oO6GBz. I'm undecided on the usefulness of these; seems like a lot of control to give up in exchange for writing slightly less code, assuming that we're inevitably going to bring in a charting library eventually.
This is not data science related, but kind of a fun write up: building a minimal Python command line text from scratch buff.ly/3ns1y4A
Cerberus is a library for doing data validation buff.ly/2JUmMKE Another nice example of not-reinventing-the-wheel, and an illustration of how declarative programming can make things easier.
There's something appealingly meta about using the tools of data science to investigate.... the tools of data science. Like this analysis of ~10M Jupyter notebooks buff.ly/3munTOD
This is pretty cool - a Python package for building interaction question/answer based command line interfaces buff.ly/34NCHSk I could see this being useful to deploy e.g. datavis tools to non-programmers.
Though Python is slower than some other languages, in many scenarios that doesn't matter because either (a) it's fast enough or (b) some other component in the workflow (disk IO, database, API) is even slower. buff.ly/34Fs3gj (use private window if you get a sign in wall)
Recursion and trees are intimately linked in programming; if you find yourself using recursion in data science then it's probably for dealing with treelike data. So it makes perfect sense to draw a tree to visualize it buff.ly/3h75Qgf
1
1
When you need something like range() but more powerful (e.g. for floating point numbers) take a look at np.linspace buff.ly/2Jl5Tsl
1
1
If you find yourself working in plain Python and missing pandas' cut and qcut functions, take a look at the built in bisect module buff.ly/3mkUtmt
Now this is what I call an in-depth write up for a single line of code: buff.ly/3l9NCvy everything you ever need to know about the lru_cache function decorator
1
PSA:Using scipy to generate distance matrices (including using e.g. seaborn's clustermap)? *don't* use the default euclidean distance if you have binary data - you need hamming distance buff.ly/3kQWhCk
Is it safe to save memory in pandas by reducing precision? Let's find out by looking at a thousand real world datasets drawingfromdata.com/pandas/n…
A long, complex read, but a very nice overview of making interactive charts using Bokeh + Flask. Lots of moving parts and multiple languages involved!
If, like me, you've never heard of matrix profiles for time series data, take a look at the tutorial for STUMPY which explains it very nicely buff.ly/369DAEH then go and use the package to try it out buff.ly/2Wceli8
1
2
I've posted before about getting data from Excel into pandas; if you need to manipulate Excel files directly, this write up has plenty of code samples that might make a good starting point buff.ly/3ldBrOJ
If anyone's working on a big enough project to require a data warehouse, take a look at whale for exploring it buff.ly/3oPVyV0
1
Nice write up on the very basic of regex with Python. I often find regex useful for wrangling and data cleaning at the start of a data exploration project buff.ly/2HBQ1kt
2
A very cool tool for those of us who spend most of our time in Jupyter notebooks: run black/pylint/flake8/other code QA tools on Jupyter notebooks buff.ly/2YF1Awk