Predictive Hacks

Must-Known hack for selecting columns in pandas

select columns in pandas

We are referring to the filter function of Pandas Dataframes. We have already mentioned it in our dplyr pipes in pandas post but now we will dive more into it to reveal some very useful hacks.

Let’s first create our Dataframe

df=pd.DataFrame(columns=['dummy'+str(i) for i in range(0,10)]+
             ['Billy'+str(i) for i in range(0,10)]+
             ['att'+str(i) for i in range(0,10)],data=None)

for i in df.columns:
    df[i]=[0,0,0]

df.columns
Index(['dummy0', 'dummy1', 'dummy2', 'dummy3', 'dummy4', 'Billy0', 'Billy1',
       'Billy2', 'Billy3', 'Billy4', 'att0', 'att1', 'att2', 'att3', 'att4'],
      dtype='object')

The like parameter

With this parameter, we can filter the columns were the input string is contained in their column names.

df.filter(like='Billy')
   Billy0  Billy1  Billy2  Billy3  Billy4
0       0       0       0       0       0
1       0       0       0       0       0
2       0       0       0       0       0

The regex Parameter

This is the most useful thing about this function. You can filter the columns whatever you want using regular expressions. Here are some examples:

#filter the columns starting with att and followed by 1, 2 or 3
df.filter(regex='att[123]')
   att1  att2  att3
0     0     0     0
1     0     0     0
2     0     0     0
#select all columns except one (Billy1)
df.filter(regex="^(?!Billy1$)")

   dummy0  dummy1  dummy2  dummy3  dummy4  Billy0  Billy2  Billy3  Billy4  \
0       0       0       0       0       0       0       0       0       0   
1       0       0       0       0       0       0       0       0       0   
2       0       0       0       0       0       0       0       0       0   

   att0  att1  att2  att3  att4  
0     0     0     0     0     0  
1     0     0     0     0     0  
2     0     0     0     0     0
#Select all except the columns that contains Billy
df.filter(regex="^(?!Billy)")
   dummy0  dummy1  dummy2  dummy3  dummy4  att0  att1  att2  att3  att4
0       0       0       0       0       0     0     0     0     0     0
1       0       0       0       0       0     0     0     0     0     0
2       0       0       0       0       0     0     0     0     0     0

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email

Leave a Comment

Subscribe To Our Newsletter

Get updates and learn from the best

More To Explore

connect with sql
R

How to Connect R with SQL

Need to Connect R with SQL It is common for Data Analysts/Scientists to connect R with SQL. For that reason,

[the_ad_group id="232"]
[the_ad id="2133"]