That's not strictly accurate. It's not that you shouldn't use references - the 50% rule video even explains in detail how to actually use references when still drawing from your imagination. But, it does say that you should not do strict, direct photo studies.

There's a lot of grey area when it comes to the 50% rule, and unfortunately there are not clear cut answers for every possible circumstance. In such situations, we have to come back to a couple of core questions:

• When doing the given drawing, am I concerned with the end result, or getting something out of it (a pretty picture to show off, or improvement in my skill) or am I just drawing and exploring and playing, and this happens to be the way I'm doing it today?

• Am I choosing to do this instead of something else? If so, what reason do I have not to do that other thing?

Ultimately only you understand your motivations, but I can say this - avoid getting stuck doing one type of thing for your 50% rule time. Push your boundaries and explore all kinds of things. If it means walking outside and drawing the things you see some of the time, then cool - but don't pigeonhole yourself into a single activity for the entirety of your "play" time.