How to clean a dental retainer?

Hey there,
so I’ve been wearing this night guard for a while now, and even though I make sure to soak it in a cleaning solution every single week, I’m still running into an issue. some kind of weird, "gritty" buildup has started accumulating on the outside, specifically tucked away in those little grooves where my bottom teeth sit.

I was wondering if anyone knows a way to get rid of that gunk? I really don't want to have to shell out money for a whole new guard because, honestly, this one fits my bite perfectly right now.

EDIT: Just to clarify, it’s a night guard.

That’s a good question. I don't really worry about that yellowish stuff too much. I just spot a few spots here and there, and maybe some Corega helps out a bit...