@databricks @azuredatabricks tips straight to your twitter feed! For Beginners to Advanced users with snippets and examples. #databricks by @fusionet24
At the Deltalake house 🏞️🏘️
Joined May 2022
- Tweets259
- Following15
- Followers595
- Likes87
There's a really rich #R package for databricks from databricks-labs. You should give it ago. If you've previously used Sparklyr or SparkR this is a worth while package to pick up for your platform.
I've jumped into a quick overview below dailydatabricks.tips/Tutoria…
Did you know about With Columns Renamed? You can pass in Key Vaule Pairs of
{old_name: new_name}
dailydatabricks.tips/tips/Sp…
NEW YEAR NEW YOU I get it but there's still time to think about this super awesome @databricks tip from last year
1/6 🎄 Launching #Decembricks!
24 days of @databricks tips to supercharge your data engineering.
Day 1: The hidden power of IPython events - your ticket to professional-grade notebook engineering 🚀
Check this one out seriously! The code is at the bottom
1/4 🧵 Just published a guide on creating Custom IPython Magics in Databricks!
First, we'll cover the basics:
Line magics (%) vs Cell magics (%%)
Creating custom commands
Registering your magics
3/4 This is part of the #Decembricks2024 series, where we're sharing a new Databricks tip every day of December!
4/4 Read the full guide here dailydatabricks.tips/tips/No…
#Databricks #DataEngineering #Python #Ipython #Spark
🔄 Getting the INVALID_STATE.RESTORATION_PERIOD_EXPIRED error in #Databricks when trying to undrop tables? Even within the 7-day window? Here's what you need to know...
🔍 Pro tip: Use 'SHOW TABLES DROPPED' to get the exact table ID and drop time. Sometimes the issue isn't about timing - it could be naming conflicts or incorrect table references. #DatabricksSQL #DataOps 2/3
💡 Quick fix: Try 'UNDROP TABLE WITH ID' instead of table name. And if you've got naming conflicts, use 'ALTER TABLE RENAME' first to clear the path for restoration. #TechTips #DatabaseAdmin 3/3
dailydatabricks.tips/tips/Er…
Ever fancied using a managed identity for your #Azure #Databricks workspace that already exists and to start using it only requires a few lines of code?
Find out how to use this from your Managed Resource Group dailydatabricks.tips/tips/Id…
🧵 Dynamically Loading Unit Tests in Databricks Notebooks 🚀
1/ Tired of the hassle of manually maintaining test suites?
Want to keep your tests closer to your code?
Our latest tip dives into dynamically loading unit tests using Python's unittest library. 🐍🧪
3/ We break down the process step by step, from setting up the TestSuite to filtering and loading the discovered tests. 📝 Code samples included! 💻
4/ Ready to transform your Databricks testing workflow? Check out the full post here: 🔗dailydatabricks.tips/tips/No… #Databricks #PythonTesting #Unittest #DynamicTesting
1/2 Are verbose window functions bogging down your @databricks SQL & PySpark code? 😰 Named window specs to the rescue! 🚀
2/2 By defining a window spec once & referencing it by name, you can:
🔹Reuse complex definitions
🔹Improve query readability
🔹Avoid errors
Check out the blog for a quick guide with examples! 🔗 dailydatabricks.tips/tips/Da…
#DataWithDatabricks #SparkTips
💡#Decembricks Day 2
Use #DeltaLake's custom metadata to:
- Track data quality metrics
- Maintain audit trails
- Document pipeline versions
- Monitor data transformations
Full guide with code examples below 👇
@Databricks #DataEngineering #Databricks
