Unpairing a remote?

How do I 'unpair' a remote from my audio receiver?

I tried unpairing the remote altogether then repairing it, but control of the audio receiver comes right back.

It is nice at times to control everything from one remote, But Not at 6am! When all that you want to do is quietly watch the news.