Why we need this?

Manually copy-pasting data from one csv to another is a tedious and time-consuming work. If you don’t have too many files then it’s fine but imagine if you have 100+ files to merge, then manual copy-pasting is not a good  option because doing same thing again and again is tedious and chances of errors increase.

Python Script

If all csv files have the same structure (same headers), use this tiny Python script to do the work. To run this script, we need python.Click here to install python

Note: Change “csv_folder” to your desired working directory.
import os
import glob
import pandas as pd
os.chdir("csv_folder")
extension = 'csv'
all_csv_files = [x for x in glob.glob('*.{}'.format(extension))]
combined_csv = pd.concat([pd.read_csv(file, encoding = "ISO-8859-1") for file in all_csv_files ])
combined_csv.to_csv( "Combined_CSV.csv", index=False, encoding='utf-8-sig')

Find code on Github


Leave a Reply

Your email address will not be published. Required fields are marked *