Import CSV file and ignore columns

[expired user #10472]'s profile image [expired user #10472] posted 7 years ago in General Permalink

Hello,

I have around 400 CSV files (70 GB) and want to import them into my database.

The CSV files have more columns than I need. So is it possible to ignore some of the columns? I can change the order in the table structure, but can't find a setting to ignore colums.

It would be a big effort to edit each file and remove the columns I don't need. Or are there any programs for Windows to edit multiple CSV files automatically?

bitseeker's profile image bitseeker posted 7 years ago Permalink

Hi Hamster,

for replacing/reordering/... in multiple files and with great regex functions you could use "dnGREP" on github. It´s very powerfull.

greets

ansgar's profile image ansgar posted 7 years ago Permalink

Another alternative: create the columns in the target table, anyway if you will use them or not. You could even set these to NULL after the export to minimize disk usage.

Please login to leave a reply, or register at first.