Filter checkins by date in Mongo instead of in Ruby - #875
Open
suttondemlong wants to merge 1 commit into
Open
Conversation
CheckinsController#index called .select with a block on the criteria, which is Enumerable#select rather than Mongoid::Criteria#select. That loaded every checkin the user had ever created, along with the eager loaded associations for all of them, and then threw away everything outside the requested day. Cost grew with account age on the endpoint the clients hit to render a single day. Use the existing by_date scope over the day's bounds so the query is served by the existing index(date: 1, encrypted_user_id: 1), and materialise with to_a so the criteria is not enumerated once per serializer pass. Measured with a user holding 60 days of checkins: before, one unfiltered find returning 60 documents; after, one filtered find returning 1, with the same total number of Mongo commands. Fixes rubyforgood#874 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #874.
The bug
CheckinsController#indexcalled.selectwith a block on the criteria. That isEnumerable#select, notMongoid::Criteria#select, so it forced the criteria to load: every check-in the user had ever created came back from Mongo, along with the eager loaded associations for all of them, and everything outside the requested day was then thrown away in Ruby.This is the endpoint the web and native clients hit to render a single day, so the cost grew with account age on one of the hottest paths in the app — and it grew fastest for the longest-tenured users.
The fix
Use the
by_datescope that already exists onCheckinover the requested day's bounds, so the query is served by the existingindex(date: 1, encrypted_user_id: 1). Both the scope and the index were already there and simply were not being used here.The resulting query:
Why the
.to_aIt looks like a no-op and isn't, which is why it carries a comment. Rendering the bare criteria makes AMS enumerate it more than once, re-running both the query and the eager loads. Measured against a user with 60 days of history, counting actual driver commands:
checkinsfindsselect {}(before)by_datecriteriaby_date+to_aSo
to_akeeps the command count identical to the original while moving the filter into the database.Test
The existing specs passed against the buggy code — the output was correct, just expensively obtained — which is how this survived. A behaviour-only test cannot catch it, so
spec/support/mongo_commands.rbadds a small driver-monitoring subscriber, and the new example asserts there is exactly onecheckinsfind and that it carries a$gte/$ltedate filter. It fails if Ruby-side filtering returns or theto_ais dropped.Behaviour note
Date.parseused to be called inside the block, so an invaliddateparam returned200 []for a user with no check-ins but raised for everyone else. It now raises consistently (422 in production viarescue_from "Exception"). More consistent, but it is a change, so worth calling out.Verification
Full backend suite green (316 examples, 0 failures),
standardrbclean,erblint --lint-allclean.One unrelated thing noticed while verifying:
spec/models/food_spec.rb:35is flaky on master (1 in 20 full-suite runs for me).Food.ftsorders byts_rank_cd(...) DESCwith no tie-breaker, and the spec asserts an exact order for two rows that rank equal. It passes 25/25 in isolation and only varies in full-suite runs. Not touched here and not caused by this change — measured at 1/20 on master and 0/20 on this branch — but mentioning it in case CI goes red on it.