Which module are you in when you encounter the 3-4 seconds 'Loading' delay?
If it's the Library module, that might indicate that you are only building "minimal" previews during import, which means that a standard-size preview will need to be built when you first load an image into the Loupe view. I routinely build 1:1 previews on import, so in Library loading images, even zoomed to 1:1, is instant. Even building standard-size previews on import should eliminate any "loading" delay at Fit/Fill in Loupe (assuming your standard preview size is set correctly in the catalog settings).
If the delay is in the Develop module, then really that is to be expected. Develop module doesn't use the existing previews, instead a new preview is built every time the Raw image is selected. Some of that rendering may have already been done and stored in the ACR cache, but even so you can still expect to see the "loading" indicator for a few seconds (maybe more, depending upon the size of the Raw file). You can turn off the "loading" indicator in the View Options menu....it doesn't actually change the speed at which the preview is rendered, but you should be able to start working on the image as soon as the sliders are "activated", which may well be sub-second.....thus giving the feeling of a snappier system.
