Introduction
As data scientists, we love to do our job efficiently without reinventing the wheel. Tips-and-tricks articles provide snippets of code for common tasks in the data science world. In this article, we’ll cover mainly Python and R, as well as other tips in Unix, Excel, Git, Docker, Google Spreadsheets, etc. Here, we will gather 10 tips and trips from our Tips Section. Stay tuned!
Python
1. How to sort a list of tuples by element
Let’s say I have the following list:
l = [(1,2), (4,6), (5,1), (1,0)] l
[(1, 2), (4, 6), (5, 1), (1, 0)]
And I want to sort it by the second element of the tuple:
sorted(l, key=lambda t: t[1])
[(1, 0), (5, 1), (1, 2), (4, 6)]
2. How to flatten a list of lists
Assume that our list is:
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
We want to flatten it into one list. We can use list comprehensions, as follows:
[item for sublist in l for item in sublist]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
3. ‘elif’ in list comprehension
Scenario: You’re dealing with product reviews that take values from 1
to 5
, and you want to create three categories:
- Good, if the review is greater or equal to
4
- Neutral, if the review is a
3
- Negative, if the review is less than
3
x = [1,2,3,4,5,4,3,2,1] ["Good" if i>=4 else "Neutral" if i==3 else "Bad" for i in x]
['Bad', 'Bad', 'Neutral', 'Good', 'Good', 'Good', 'Neutral', 'Bad', 'Bad']
4. A shebang line: #!/usr/bin/python3
In many .py
files, we may see the shebang line at the top of the script. Its purpose is to define the location of the interpreter. By adding the line #!/usr/bin/python3
at the top of the script, we can run the file.py
on a Unix system, and it’ll automatically understand that this is a Python script. Alternatively, you could run the script as python3 file.py
. For example, assume the file.py
is:
#!/usr/bin/python3 print("Hello shebang line")
And we can run on Unix as:
$ ./file.py
R
6. Joining with ‘dplyr’ on multiple columns
dplyr
allows us to join two data frames on more than a single column. All you have to do is to add the columns within the by
like by = c("x1" = "x2", "y1" = "y2")
. For example:
library(dplyr) set.seed(5) df1 <- tibble( x1 = letters[1:10], y1 = LETTERS[11:20], a = rnorm(10) ) df2 <- tibble( x2 = letters[1:10], y2 = LETTERS[11:20], b = rnorm(10) ) df<-df1%>%inner_join(df2, df2, by = c("x1" = "x2", "y1" = "y2")) df
# A tibble: 10 x 4 x1 y1 a b <chr> <chr> <dbl> <dbl> 1 a K -0.841 1.23 2 b L 1.38 -0.802 3 c M -1.26 -1.08 4 d N 0.0701 -0.158 5 e O 1.71 -1.07 6 f P -0.603 -0.139 7 g Q -0.472 -0.597 8 h R -0.635 -2.18 9 i S -0.286 0.241 10 j T 0.138 -0.259
7. How to store models in R with a for loop
Let’s say that we want to run a different regression model for each Species
in the iris
data set. We can do it in two different ways, as follows:
Store the models in a list:
my_models<-list() for (s in unique(iris$Species)) { tmp<-iris[iris$Species==s,] my_models[[s]]<-lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width, data=tmp) } # get the 'setosa' model my_models[['setosa']]
Call: lm(formula = Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, data = tmp) Coefficients: (Intercept) Sepal.Width Petal.Length Petal.Width 2.3519 0.6548 0.2376 0.2521
Store the models by name using assign
:
for (s in unique(iris$Species)) { tmp<-iris[iris$Species==s,] assign(s,lm(Sepal.Length~Sepal.Width+Petal.Length+Petal.Width, data=tmp)) } # get the 'setosa' model get("setosa")
Call: lm(formula = Sepal.Length ~ Sepal.Width + Petal.Length + Petal.Width, data = tmp) Coefficients: (Intercept) Sepal.Width Petal.Length Petal.Width 2.3519 0.6548 0.2376 0.2521
8. How to pass multiple parameters to ‘sapply’
Assume that we want to run an sapply
or lapply
in R, and the function has multiple parameters. Then we can define the parameter in which we want to apply the sapply
and assign fixed values to the rest:
# this is the function like a linear equation # of the form y= a + b * x my_func<- function(a,b,c) { a+b*c } # the values of the x x = c(1,5,10) # we set a=1 and b=2 sapply(x,my_func,a=1, b=2)
[1] 3 11 21
9. How to get the column of the max value by row
Assume that our DataFrame is:
set.seed(5) df<-as.data.frame(matrix(sample(1:100,12),ncol=3)) df
V1 V2 V3 1 66 41 19 2 57 85 3 3 79 94 38 4 75 71 58
We can get the index and the name of the max column by row, as follows:
colnames(df)[max.col(df,ties.method="random")]
[1] "V1" "V2" "V2" "V1"
10. How to generate random dates
We can generate random dates from a specific range of Unix time stamps using the uniform distribution. For example, let’s generate 10 random dates:
library(lubridate) lubridate::as_datetime( runif(10, 1546290000, 1577739600))
[1] "2019-12-09 15:45:26 UTC" "2019-08-31 19:28:03 UTC" "2019-01-13 12:15:13 UTC" "2019-11-15 00:13:25 UTC" [5] "2019-01-19 06:31:10 UTC" "2019-11-02 12:46:34 UTC" "2019-09-04 19:16:31 UTC" "2019-07-29 11:53:43 UTC" [9] "2019-01-25 23:08:20 UTC" "2019-02-03 02:30:21 UTC"