If it's a "latino" person calling you Boss or Jefe, you're overreacting and over-analysing the thing.
It's a cultural thing. Here in South America for example, it's cool and normal to be called Jefe or Capo or Maestro. I get Capo all the time from random people, and I'm not even from the Mafia.
It also bothered me and really annoyed me when I first emigrated to South America from Europe. Then after a while, I understood that Latino people are just way more friendly, laid back and relaxed than serious and stern people from European or Anglo Saxon countries. They mean no harm at all, the Latinos, no disrespect with these words. We're just too stiff-collared to get it, at first.
So my advice would be, try to learn to accept it in a relaxed manner. The meaning of these words are actually positive, take it lightly, don't be dour European about it.
On a funny note, Capo is used in a cool, endearing way by South American men, but in my dour native European country, it's an insult meaning, a warden at a Nazi concentration camp. So, at first it startled me to be called Capo, but I understood the innocent Latin meaning of it.