I finally got my Python script to read my messy CSV files

I spent Saturday afternoon trying to get a script to import my moving company's old inventory lists. The files have extra commas in the item names and random blank lines, so my first 3 attempts just crashed. I finally figured out I needed to use the csv module with a custom dialect instead of just splitting on commas. It took about 2 hours of trial and error in my kitchen, but now it loads 400 rows without a single error. Has anyone else hit weird formatting issues with data exports from old software?

1 comments

1 Comment

angela58721d ago

That reminds me of something I read last week about old accounting software from the 90s still being in use at some warehouses. The article said those systems export data with like 8 different date formats and sometimes add a hidden character at the end of every line. It made me glad I don't deal with that stuff anymore. Sounds like your custom dialect was the right fix though. I bet half the battle was just figuring out the pattern of the mess.