I know what you are talking about.
I wear my gliders in my sweatshirt for a few hours each day and sometimes if their nails are sharp they can tickle me pretty bad lol! This is what I do:
I change into a cami (with or without a built in bra) and then I put a sweatshirt or jacket over top. I then tuck in the bottom of the sweatshirt underneath my cami so my gliders can't crawl out. When I get them into my sweatshirt, I take one at a time out of their sleeping pouch and tuck them underneath my cami so they are next to my skin and bra. One of my gliders, Kiwi, will hold onto my bra and sleep against my chest for the entire time I have them there (she is actually there now hahaha) and my other glider will sometimes climb out and sleep between my sweatshirt and cami. So kiwi is my "bra baby." Maybe your glider is a bra baby and would rather be sleeping there than in your sleeve. Just a suggestion;)