Speed up request response with cache or binary response
I want to speed up the response of a request with a cache or binary response
It takes more than 30 seconds ~ 500 seconds to display json response for 5k ~ 100k records.
How can you make it faster?
Since the response time is still slow when rendering
Is it possible to make a binary answer (skip visualization)?
WeatherLog Load (335.5ms) SELECT "weather_logs".* FROM "weather_logs" ORDER BY "weather_logs"."datetime" ASC LIMIT 90000
Write page /Users/public/index.json (35.2ms)
Completed 200 OK in 40488ms (Views: 32459.3ms | ActiveRecord: 336.8ms)
controller
caches_page :index
respond_to :json
def index
begin
@weather_logs = WeatherLog.result(q)
respond_to do |format|
render json: @weather_logs
return
end
rescue Exception => e
respond_to do |format|
format.json { render json: {status: "Invalid Request #{e.backtrace.first(3)}"} }
end
end
end
Dataflow: Step 4 is the bottleneck for rendering
Shortly speaking
I only need to send a response to the client within a short time,
Regardless of the format.
Even the binary format is fine.
Please give me a direction or idea to improve productivity.
source to share
This is definitely just rendering, which is an issue, which is the result of WeatherLogs.result (q). Have you compared the results of the WeatherLogs # queries? How long does it take to iterate over this collection returned versus rendering it? Are there any methods called when passing it JSON, eg. not just class fields?
If this is definitely rendering, which is an issue you might want to consider using the LRU cache when inserting data into WeatherLogs, substantially denormalizing the data requested by clients. Because, depending on your traffic and the number of possible requests (although this will also affect denormalized data), it will almost never hit the cache when it is generated at render time. This article might be a good start: http://oldblog.antirez.com/post/redis-as-LRU-cache.html
But without understanding why rendering takes so long, I would not go as far as building a complex cache if JSON rendering is an OJ problem, which could be a solution, or any other lib that claims to be the fastest, but I have do not assume this is not a problem.
Happy help if you have further questions.
source to share
You must explicitly call Oj:
def index
begin
@weather_logs = WeatherLog.result(q)
respond_to do |format|
render json: Oj.dump(@weather_logs)
return
end
And if you want to make it implicit, try this gem from my company: https://github.com/GoodLife/rails-patch-json-encode
source to share