Tuesday, April 18, 2006

DOS vs UNIX line breaks

I was writing a shell script to parse some data in csv format generated by Microsoft Excel and was wondering why the script didn't work as I expected.

I took a look at the .csv file using more command and it looked OK to me...

Half an hour passed by and I still cannot figure the problem out, I used vi to open the csv file and saw that there were additional ^M characters everywhere in the file.

After googling around for a while, I found that the ^M's were there because the csv file contained "carriage return (\r)" characters, which is the default line break character in Windows, instead of "new line (\n)" characters as it should be. To fix this problem, I have to execute the command
tr "\r" "\n" < inputdata.csv > gooddata.csv
to remove all ^M's and after that my script worked just fine.

1 Comments:

At 6:26 PM, Anonymous Anonymous said...

AKAIK, there's a utility called dos2unix that can convert text file from DOS into UNIX format.

 

Post a Comment

<< Home