You touched on something I read about recently (can't remember where). It was about how geographic isolation in high achieving youth can sometimes lead to issues where one's identity can get tied so closely to high achievement that one may be more hesitant to test oneself on a broader scale because the risk of failure can inherently risk a greater portion of that person's identity.
The example is like if a kid is used to always being the "smart one" in his corner of the world, he may not grow up with as much exposure not only to those smarter than him, but also the failure/retrial cycles of development in himself or others. So, he may be subconsciously more hesitant to take the risks necessary to test himself by maybe attending a prestigious college (for example) lest he find out he isn't as smart as he thought when the scale is larger. He'd no longer the "smart one". He'd just be "normal" on that scale. Such "failure" could threaten more of his identity than it might for another person who has had more exposure to such "failure" and thus doesn't tie it as closely with his/her identity. Instead, they view it as a normal occurrence.
Whether or not I agree with what you're saying, it's a related theory. Competing for an FCS Championship is a huge part of our program's identity. If that is replaced with being a solid program that can "only" aspire to conference championships and bowl games in a bigger, better pool, it threatens that identity. Accordingly, many would rather be the 50th or 75th best CFB team but have the banner than risk being "normal" in the better pool.