Data Engineer, car nerd, 70s rock n roll, Archmagos Logis
Joined August 2013
- Tweets2.4K
- Following463
- Followers5.9K
- Likes889
I have experienced every one of these. 100% true.
>be senior devops engineer in Germany
>master's degree, 10 years experience, five interview rounds
>72,000€ before tax
>C2 German required
>All team members based in Asia speaking English
>mandatory office attendance to sit on Teams with people at another mandatory office
>"we have a startup mindset"
>company founded in 1894, CEO is the founder's great-grandson
>"we have flat hierarchies"
>eight people can veto your decision, none can approve it
>hired to "drive digital transformation"
>need department head's signature to install Docker
>service account request rejected because "accounts must belong to a person"
>CI/CD pipeline ends with a ticket for Jürgen
>Jürgen on holiday until the 23rd
>AI tools forbidden because Datenschutz
>database password in an encrypted Excel file called Passwörter_final_NEU.xlsx
>root password in Confluence
>on-call every weekend
>company phone listed under benefits
>ask for a raise
>unfortunately the salary band is exhausted
>no budget for a second monitor
>external consultant gets 1,400€ a day to ask you how the code works
>take banana from company fruit basket
>office manager says those are for the upcoming customer visit
>Fachkräftemangel
Data Modeling is so overrated.
All these people selling their modeling courses and books to Data Engineers and Data Analysts.
Telling learners that it's so difficult.
1. Learn the basics (it's not that difficult)
2. Feed AI with good context (goals & data)
3. Let AI do the modeling
4. Use your skills to find errors it did and help it correct
5. Ask it performance questions to find out optimization possibilities.
Be clever and use AI as a lever to make you more efficient.
It has been trained on this shit. So it knows.
You just need to steer it.
@hashnode must be the dumbest platfrom I've ever seen. Doesn't even let me post in my own publication.
All I wanted to share was:
6 Free Data Engineering courses I created for you
Don't want to bore you with "bla bla bla" for my first post, so real quick:
Here are 6 free courses that I created specifically for Data Engineers.
- Python for Data Engineers
- Docker Fundamentals
- Databricks Declarative Pipelines
- DuckDB and MotherDuck
- Kestra Workflow Orchestration
- Exasol High Performance OLAP Data Analysis
I bundled them together for you here: learndataengineering.com/p/f…
More are coming.
If you have any career questions let me know.
I'm happy to help!
Andreas
6 Free Data Engineering Courses I created for you:
- Python for Data Engineers
- Docker Fundamentals
- DuckDB & MotherDuck
- Kestra Orchestration:
- Databricks
- Exasol OLAP
--> learndataengineering.com/p/f…
These people are lying to you: "It's just 5 steps to becoming a Data Engineer, trust me bro".
It's never that easy.
I made over 30 courses on this shit, worked for 10+ years as a Data Engineer.
Still feel like a beginner.
🔧 Road to Data Engineering, should be:
Stage 1: Python + Github (SW dev basics)
Stage 2: SQL & Relational Databases (incl. Modeling) Stage 3: Infrastructure (Servers, Networking basics) Stage 4: Docker (host your python scripts)
Stage 5: API basics (you'll need them use FastAPI)
Stage 6: Data Warehouses + Modeling (learn with DuckDB)
Stage 7: Data Lakes (use AWS S3)
Stage 8: Orchestration (e.g. Airflow, Kestra)
Stage 9: Spark (Batch processing)
Stage 10: Kafka & Spark streaming
Stage 11: Find alternatives on AWS for all the services you used so far and create the same thing there. 🏆 Data Engineer unlocked
+ wrap all this into a personal project you can use for your portfolio
📊 Data Engineer
Stage 1: Python
Stage 2: SQL
Stage 3: Git + GitHub
Stage 4: Linux
Stage 5: Data Modeling
Stage 6: ETL / ELT
Stage 7: Apache Spark
Stage 8: Airflow
Stage 9: Kafka
Stage 10: Data Warehouses (Snowflake / BigQuery)
Stage 11: Cloud Data Pipelines
Stage 12: Build Scalable Data Platforms
🏆 Data Engineer unlocked 🙌
X Fels like the only platform that encourages you to post multiple times a day and share your raw thoughts.
Unlike LinkedIn and Instagram. They hate it if you post more than once a day.
LinkedIn is full of AI slop. It got kinda boring.
Who's with me?
This is exactly what's going to happen.
It's actually already happening. Who talked about pricing 6 months ago? One year? Basically nobody
Dass die Tür vom neuen X5 erst beim vierten mal antatschen auf geht ist genau das was ich erwartet habe. Mega Lösung BMW 😂
Aber Hauptsache alle immer über Tesla lästern
Teachable is currently down so learndataengineering.com doesn't work.
Sorry for the inconvenience! I hope they fix it soon.
#SaveStargate let's go!
Hey @AmazonMGMStudio, @PrimeVideo, and @JeffBezos. Do you see how passionate Stargate fans are? ⤵️⤵️
We've been fighting for years to bring back Stargate, and we're way stronger in numbers and spirit than anyone realizes.
All you have to do is re-greenlight Martin Gero's Stargate show... and all these millions of fans will have your back again! #SaveStargate
deadline.com/2019/12/we-want…
