I couldn't find any information on sugar gliders eating anchovies, so personally I wouldn't feed it.
I did, however, find that people give their gliders fish (and shrimp) as a protein source/treat. Apparently, they like white fish and salmon, but make sure that it's fresh fish (for example, not canned tuna as it contains preservatives).
If you really want to try anchovies out, I recommend making sure there's nothing added to them like preservatives or artificial sweeteners. Check to make sure it's not too high in fat, and definitely no bones.
In the wild they would never go near a body of water since they can't swim... so no fish. But, our babies aren't wild so of course they tend to get foods they wouldn't in the wild.