How to drag a selection to crop? I must be doing something wrong...

I'd like to manually click and drag a selection and have the crop handles snap to my selection.

This works ONLY if my starting coordinate is outside the initial crop handles.   However if my initial coordinate is INSIDE the crop handles, clicking and holding starts moving the default crop.   If I hold the SHIFT key down, it kinda works but it locks the aspect ratio.

Am I missing something?   My workflow is very inefficient wrt cropping because of this.

Thanks.