OK checking :
just tried it witha different folder (RAW_BU, 549 images)
Filters off, click flags onto two images, >sort by pick..nothing.. try swapping the A/Z to Z/A ..aha! There I have three flagged pics at the top.
Try sort b y rating.. needs to be Z/A to get the highest rated to the top
add another flag and return to sort by pick, to see what happens..OK they seem to appear OK- A/Z slings me to the bottom of the page, Z/Aback to te top - so far so good..(except it's the reverse of what I wuld expect i.e. A/Z would put rated at top -oh well)
Trying sort by rating - whizzes me to middle of selection..i.e. neither top nor bottom.. can't see the logic in that ( doesn't mean there isn't any

) aha .. it might be to whatever a previously selected image was..?
an experiment:
All using A/Z -
Try capture time - zooms me
almost to top
Try Import order zooms me to 3/4 way to bottom
Try Edit time and edit count: zoom me to top of page
Try rating- zooms 2/3rds to bottom
Try PIck again zooms to top, but has remembered the Z/A orientation itself
The others zoom me to anywhere but directly to top or bottom
No idea what that means.. but just thought I'd try it
Back to quick collection, nope, flags are all over the area when sort by pick is selected, either A/Z or Z/A