This may sound silly, SORRY !!!!!!
You haven't just got two filters (flagged buttons) clicked on at the same time?
Now I'm just wondering, eliminating scenarios,
If you change the metadata column to show flags, are you presenting the correct results by clicking on, flagged, unflagged, or...