Clever Geek Handbook

Starting a rails request in batches

I have a table A(:name, :address, :phone)

with 500,000 records. I want to run this query:

johns = A.where(:name => "John")

This query should return 150,000 results. But the execution of this query gives me this result: Killed

.

How do I rewrite this query so that the query runs in batches of 1000 in the database?

+3


source to share


3 answers


You need to use find_each

with the batch_size option.



A.where(:name => "John").find_each(batch_size: 1000) do |a|
  # your code
end
+5


source


An alternative to using find_each

is to use find_in_batches

.

The difference in this case is - find_each

will give your block each element and will pass through your batch element by position. find_in_batches

will deliver your batch of elements to an array in your block.

I assumed your model A

is actually called Address

. You can do something like this:



Address.where(name: "John").find_in_batches(batch_size: 1000) do |addresses|

  # Your code that you might want to run BEFORE processing each batch ...

  addresses.each do |address|
    # Your code that you want to run for each address
  end

  # Your code that you might want to run AFTER processing each batch ...

end

As you can see, this gives you a little more flexibility in how you handle your batch processing. However, if your needs are simple, just stick with it find_each

.

0


source 


 A.where(:name => "John").find_each(batch_size: 1000) do |a|
    # your code
 end
0


source






More articles:


All Articles