Mass data creation with factories - engage the warp drives!

  • Thu Mar, 2021

  • 21 views

Creating data using factories is a great convenience when working with seeding test databases - but it's very slow for big datasets you might want to create. Let's speed it up!

Let's take for example a simple factory create command and run it 1000 times. We'll do it in an artisan command we created for the purpose of creating all this data so we can track times, but the gist is:

Model::factory()->count(1000)->create();

Our commands output:

$ php artisan create:data 1000

Adding 1,000 Rows of data

✔ Done (20.70049s)

Done in 20 seconds. We can do better.

Inserting one by one into the database - which is the default method when factory creating models - is slow. Lets batch insert some models:

Lets checkout what we've done here.

  • We've create a set of chunks we need to go through
  • For each of these chunks, we've used the factory method for our model to make some data for us
  • We've converted these directly into an array
  • We've inserted them all using just 1 database insert query
  • We've then done this for however many chunks we need to do!

What sort of execution time do we get now?

$ php artisan create:data 1000

Adding 1,000 Rows of data

✔ Done (0.50125s)

BIIG gains if you look in the right places!