@databricks @azuredatabricks tips straight to your twitter feed! For Beginners to Advanced users with snippets and examples. #databricks by @fusionet24
At the Deltalake house 🏞️🏘️
Joined May 2022
- Tweets259
- Following15
- Followers595
- Likes87
There's a really rich #R package for databricks from databricks-labs. You should give it ago. If you've previously used Sparklyr or SparkR this is a worth while package to pick up for your platform.
I've jumped into a quick overview below dailydatabricks.tips/Tutoria…
Did you know about With Columns Renamed? You can pass in Key Vaule Pairs of
{old_name: new_name}
dailydatabricks.tips/tips/Sp…
NEW YEAR NEW YOU I get it but there's still time to think about this super awesome @databricks tip from last year
1/6 🎄 Launching #Decembricks!
24 days of @databricks tips to supercharge your data engineering.
Day 1: The hidden power of IPython events - your ticket to professional-grade notebook engineering 🚀
Check this one out seriously! The code is at the bottom
4/4 Read the full guide here dailydatabricks.tips/tips/No…
#Databricks #DataEngineering #Python #Ipython #Spark
3/4 This is part of the #Decembricks2024 series, where we're sharing a new Databricks tip every day of December!
2/4 Then, practical examples including:
Cluster information fetching
SQL execution plan analysis
Notebook context monitoring
1/4 🧵 Just published a guide on creating Custom IPython Magics in Databricks!
First, we'll cover the basics:
Line magics (%) vs Cell magics (%%)
Creating custom commands
Registering your magics
💡 Quick fix: Try 'UNDROP TABLE WITH ID' instead of table name. And if you've got naming conflicts, use 'ALTER TABLE RENAME' first to clear the path for restoration. #TechTips #DatabaseAdmin 3/3
dailydatabricks.tips/tips/Er…
🔍 Pro tip: Use 'SHOW TABLES DROPPED' to get the exact table ID and drop time. Sometimes the issue isn't about timing - it could be naming conflicts or incorrect table references. #DatabricksSQL #DataOps 2/3
🔄 Getting the INVALID_STATE.RESTORATION_PERIOD_EXPIRED error in #Databricks when trying to undrop tables? Even within the 7-day window? Here's what you need to know...
Ever fancied using a managed identity for your #Azure #Databricks workspace that already exists and to start using it only requires a few lines of code?
Find out how to use this from your Managed Resource Group dailydatabricks.tips/tips/Id…
4/ Ready to transform your Databricks testing workflow? Check out the full post here: 🔗dailydatabricks.tips/tips/No… #Databricks #PythonTesting #Unittest #DynamicTesting
3/ We break down the process step by step, from setting up the TestSuite to filtering and loading the discovered tests. 📝 Code samples included! 💻
2/ The key? Reflection magic! ✨ By introspecting the notebook module, we can automatically discover and load test cases. No more manual updates! 🙅♀️
🧵 Dynamically Loading Unit Tests in Databricks Notebooks 🚀
1/ Tired of the hassle of manually maintaining test suites?
Want to keep your tests closer to your code?
Our latest tip dives into dynamically loading unit tests using Python's unittest library. 🐍🧪
2/2 By defining a window spec once & referencing it by name, you can:
🔹Reuse complex definitions
🔹Improve query readability
🔹Avoid errors
Check out the blog for a quick guide with examples! 🔗 dailydatabricks.tips/tips/Da…
#DataWithDatabricks #SparkTips
1/2 Are verbose window functions bogging down your @databricks SQL & PySpark code? 😰 Named window specs to the rescue! 🚀
