Convert A Massive CSV To Many CSVs Using PHP

One CSV Many CSVs
I’m in the process of working in a CSV file with roughly 26000+ rows and 20 some odd columns. Each row represents an historical purchase order, including information such as date, time, name, address, product name, product sku, etc. My end goal is to import all these transactions into an existing e-commerce system. The trouble is, is that my import tool can only process 1500 rows in one go, or I get connection reset errors from the server.

The solution here is to break up the master CSV file into discreet 1500 row chucks and save in separate files. So rather than hack my way through Excel, copying and pasting 1500 row chunks into new workbooks (ug…) I decided to write me a little PHP script to do the job. It took about 30 minutes. Not only is this a faster way to breakup these transactions, it is completely error free. There’s a high likelihood I would have missed or duplicated some rows having had done this manually.

Current code assumes your master file is in the same directory as the script. Adjust as necessary.


@ini_set( 'display_errors', 1 );


echo 'start <br/>';

$file_name_base = 'masterfile';
$master_file = "{$file_name_base}.csv";

$fh = fopen( $master_file, "r" );
if( $fh ) { // valid file?

	$i = 0;
	$records_per_file = 1500;
	$header_row = null;

	// loop through csv rows
	while ( ( $row = fgetcsv( $fh, 0, ',' ) ) !== false ) {
		if( $i == 0 ) { // first row?
			// save column names row
			$header_row = $row;

		if( $i % $records_per_file == 0 ) { // time to create a new file?

			// some vars, duh
			$curr_file_name = "{$file_name_base}_{$i}.csv";
			$curr_fh = fopen( $curr_file_name ,"w" );
			if( isset( $header_row ) && !empty( $header_row ) && $i > 0 ) { // we cool?
				// yes, add header row
				fputcsv( $curr_fh, $header_row );
			echo "{$i} - create new file: {$curr_file_name}. <br />";
		fputcsv( $curr_fh, $row );
	echo $i . '<br/>';


Posted in: Code Samples, Development  |  Tagged with: , ,  |  Leave a comment

Leave a Reply

Most recent work

  • Reductress Education Online satire workshops from the one and only Reductress.
  • Douglass Interior Products Since 1978, Douglass Interior Products has been providing exceptional materials for the aircraft, marine, automotive and interior design industries.
  • Betty Mills Betty Mills is an e-commerce store offering cleaning, sanitary & medical supplies.
  • Reductress I crank out on-going web projects for Reductress. Super fun as you can imagine.

Need hosting?

  • Media Temple My go to hosting provider. You get rockstar hosting and I make a little cash. No pressure, just clean honest fun.

Past work

  • How To Win At Feminism The badasses over at Reductress are at it again with their new book How To Win At Feminism.
  • Nice Manners Music Nice Manners is an LA based recording studio offering their clients all sorts of digital audio services. This is a demo of an internal genre based music search & player I built for them. I am ecstatic about launching this project—check it out.
  • Our Restroom Our Restroom is an (inter)national crowdsourced campaign who’s goal is simple: compel businesses to make their single occupancy restrooms gender neutral. Championed by Kristin Russo, designed by Allison Weiss, built by me on WordPress.
  • Get Your Shit Together This is a site dedicated to helping you get your shit together before an unexpected tragedy, like the loss of a loved one.
  • IHG Instagram Sweepstakes An awesome Instagram driven sweepstakes site for IHG.
  • Trivial Beersuit A site that provides professional pub trivia to bars and restaurants.
  • Audi YouTube Channel This site was a quick turn-around hair pulling adventure for the Super Bowl: revamping Audi’s YouTube Channel.
  • Everyone Is Gay Everyone Is Gay is an indispensable question and answer resource for the LGBTQ community. Hell, it’s great advice for anyone with a body with an emotion or two. I made the site responsive among other things. Give it a look see.
  • The new Providing tools to help students become teachers.
  • Valhalla DSP A one man Seattle based company, writing pretty sick professional audio plugins. Check ’em out!
  • Christine Chaney Creative The online presence of Seattle artist Christine Chaney. Specializing in architecture, art and apparel.
  • Holiday Inn Tumblr A quick, fully responsive update to the Holiday Inn Tumblr page.
  • Microsoft's Art of Touch An artful HTML5 Microsoft website. Sadly, this site is no longer live.
  • Microsoft Hardware Healthy Computing Tool A useful little tool to help you find the perfect Microsoft mouse or keyboard.
  • Pella Professional Pella manufactures high quality doors and windows. This is their professional site.
  • Imaging the World A non-profit dedicated to providing ultrasound services in remote areas of the world.
  • Microsoft Visual Studio 2012 Launch This sweet new Visual Studio 2012 Launch website sports a mobile first, responsive web design architecture and comes in 14 different languages.
  • Ginny Ruffner: A not so still life A site for a documentary film about Seattle artist Ginny Ruffner.