Friends don’t let friends export to CSV

I worked for a few years in the intersection between data science and software engineering. On the whole, it was a really enjoyable time and I'd like to have the chance to do so again at some point. One of the least enjoyable experiences from that time was to deal with big CSV exports. Unfortunately, this file format is still very common in the data science space. It is easy to understand why - it seems to be ubiquitous, present everywhere, it's human-readable, it's less verbose than options like JSON and XML, it's super easy to produce from almost any tool. What's not to like?

Robin Kaveland

I'm not going to pretend to be some sort of expert on this matter, but even as a casual it seems CSV isn't exactly scalable to large data sets. It seems to work great for smaller exports and imports for personal use, but any more complicated matters it seems wholly unsuited for.