Manly34azoz -
Sugar Gliders are Nocturnal - they do sleep all day long and are then awake and active at night while you are sleeping.
If you are carrying your glider all day you may be frequently interrupting his sleep - think about trying to sleep when some one keeps bouncing and shaking your bed all night long.
Try letting your glider have some solid undisturbed sleep in the cage for most of the day - then carry him for a few hours in the afternoon. You may find he will wake up a bit earlier in the evening if he has slept well most of the day.