Most people who extend invitations are just seen as regular sociable types who are trying to arrange a fun get together. It's highly rare that someone gets asked to hang out and thinks, "Oh wow, this person's so desperate they've lowered themselves to inviting me to something?!? What a loser!" If anything they'll be happy or flattered to get the invitation.