Predictive Hacks

How to Concatenate Multiple CSV files in Linux

Assume that your files are my_file_1.csv, my_file_2.csv, my_file_3.csv and so on, and you want to concatenate all of them into one file. Then you can do it with the following command:

cat my_file_*.csv > merged.csv

which we generate a file called merged.csv.

How to deal with headers

Now, if your csv files have headers, and you want to keep the headers of just one file then you can use the following command:

awk '(NR == 1) || (FNR > 1)' my_file_*.csv > merged.csv

FNR refers to the number of the processed record in a single file and NR refers to all files, so we keep the first line which is the header and we ignore the first lines of each file.

Finally, if you want to remove the headers from all files:

awk 'FNR > 1' my_file_*.csv > merged.csv

Share This Post

Share on facebook
Share on linkedin
Share on twitter
Share on email

Subscribe To Our Newsletter

Get updates and learn from the best

More To Explore


Image Captioning with HuggingFace

Image captioning with AI is a fascinating application of artificial intelligence (AI) that involves generating textual descriptions for images automatically.


Intro to Chatbots with HuggingFace

In this tutorial, we will show you how to use the Transformers library from HuggingFace to build chatbot pipelines. Let’s