The Back/Forward buttons don't go back and forward between modules and images. It's strictly folders and collections.
They go back and forward among the last several folders and collections you visited. For example, if I viewed folder A, smart collection B, collection set X, and folder Y in that order, and then click the Back button four times, Lightroom would show me folder Y, collection set X, smart collection B, and then folder A. Like going back through your web browser history list, but with folders and collections.
The easy way to remember or understand this is to click the path/filename popup text to the right of the Back/Forward buttons in the Filmstrip. When you click that popup, it shows you a Recent Sources list. That's the sequence of the folders and collections you have visited. When you click Back or Forward, you are moving through that Recent Sources list. Recent Sources is the Lightroom version of web browser history.
Back and Forward have the same flaw as navigating folders and collections normally: They will not remember the specific image that was selected last time you were in there, unless the same image was also in the last source you visited. They always go to the first image.
The way Back and Forward work is definitely not obvious; it took me a while to understand what was going on. But once you get it, then it's obvious...