Not to mention the odds of them forgetting on that deep a level...a Dive like that can show even memories that have been deliberately or reflexively suppressed. Someone would have had to have tampered with both Jr. and Albedo's memories for that to happen, and even then I'm not sure it wouldn't have come up.