get rid of file names with newline character
by Pedroski from LinuxQuestions.org on (#6NZRG)
Messing around with a 5.5 million row csv file and pandas, I decided to split the csv into many csvs according to the column Site_Reference.
I got all the unique names in column Site_Reference and saved them to a text file, each unique name on its own line. There a 1775 unique names.
Then I searched the big csv and created a much smaller csv for each unique Site_Reference name. Took about 40 minutes on this laptop.
Trouble is, the first time around, I forgot to remove the newline character from the unique Site_Reference name.
Now I have what looks like 2 identical files in the folder, except, 1 file name contains a newline character. (And only the headers, because pandas never found any names ending with \n)
How can I get rid of all the files whose file name contains \n? When I paste an example here, it looks like this:
Quote:
Something in bash would be best, I think!
Code:#! /bin/bash
getPath="/home/pedro/myPython/pandas/csv/TMS/"
cd $getPath
echo "getting file names ... "
files=$(ls)
for f in $files;
do
# what to do??
rm $f
; done
I got all the unique names in column Site_Reference and saved them to a text file, each unique name on its own line. There a 1775 unique names.
Then I searched the big csv and created a much smaller csv for each unique Site_Reference name. Took about 40 minutes on this laptop.
Trouble is, the first time around, I forgot to remove the newline character from the unique Site_Reference name.
Now I have what looks like 2 identical files in the folder, except, 1 file name contains a newline character. (And only the headers, because pandas never found any names ending with \n)
How can I get rid of all the files whose file name contains \n? When I paste an example here, it looks like this:
Quote:
01B00015 .csv |
Code:#! /bin/bash
getPath="/home/pedro/myPython/pandas/csv/TMS/"
cd $getPath
echo "getting file names ... "
files=$(ls)
for f in $files;
do
# what to do??
rm $f
; done