Samantha Jackson

🐼🤹‍♂️ pandas trick #77: Want to combine the smaller categories in a Series into a single category called "Other"? 1. Save the index of the largest values of value_counts() 2. Use where() to replace all other values with "Other" See example 👇 #Python #DataScience #pandastricks

🐼🤹‍♂️ pandas trick: Want to calculate the difference between each row and the previous row? Use df.col_name.diff() Want to calculate the percentage change instead? Use df.col_name.pct_change() See example 👇 #Python #DataScience #pandas #pandastricks

🐼🤹‍♂️ pandas trick: Need to refer to a local variable within a query() string? Just prefix it with the @ symbol! See example 👇 #Python #DataScience #pandas #pandastricks

🐼🤹‍♂️ pandas trick: "loc" selects by label, and "iloc" selects by position. But what if you need to select by label *and* position? You can still use loc or iloc! See example 👇 P.S. Don't use "ix", it has been deprecated since 2017. #Python #DataScience #pandas #pandastricks

🐼🤹 pandas trick: There are many display options you can change: max_rows max_columns max_colwidth precision date_dayfirst date_yearfirst How to use: pd.set_option('display.max_rows', 80) pd.reset_option('display.max_rows') See all: pd.describe_option() #Python #pandastricks

🐼🤹‍♂️ pandas trick: Need to convert a column from continuous to categorical? Use cut(): df['age_groups'] = pd.cut(df.age, bins=[0, 18, 65, 99], labels=['child', 'adult', 'elderly']) 0 to 18 ➡️ 'child' 18 to 65 ➡️ 'adult' 65 to 99 ➡️ 'elderly' #Python #pandas #pandastricks

🐼🤹‍♂️ pandas trick: Calculate % of missing values in each column: df.isna().mean() Drop columns with any missing values: df.dropna(axis='columns') Drop columns in which more than 10% of values are missing: df.dropna(thresh=len(df)*0.9, axis='columns') #Python #pandastricks

🐼🤹‍♂️ pandas trick: Numbers stored as strings? Try astype(): df.astype({'col1':'int', 'col2':'float'}) But it will fail if you have any invalid input. Better way: df.apply(pd.to_numeric, errors='coerce') Converts invalid input to NaN 🎉 #Python #pandastricks

🐼🤹‍♂️ pandas trick: Select columns by data type: df.select_dtypes(include='number') df.select_dtypes(include=['number', 'category', 'object']) df.select_dtypes(exclude=['datetime', 'timedelta']) #Python #DataScience #pandas #pandastricks

🐼🤹‍♂️ pandas trick: Filter DataFrame by multiple OR conditions: df[(df.color == 'red') | (df.color == 'green') | (df.color == 'blue')] Shorter way: df[df.color.isin(['red', 'green', 'blue'])] Invert the filter: df[~df.color.isin(['red', 'green', 'blue'])] #Python #pandastricks