Let’s get this one out of the way first. No one over the age of 12 is calling anyone "daddy" without a tinge of something else there. If we get historical, its first use dates back to 1621, when prostitutes used it to refer to their pimps. That’s super unfortunate.
Jump ahead to the 1970s, when the gay community co-opted the word for leather subculture, with the term “leather daddy” coming to be. That term has stuck around in the leather community, and even Alex Jones, the host of InfoWars and lingual purist, has gotten in on the leather daddy action. The gay community has continued to adopt the word by dropping the "leather" part, with "daddy" alone generally meaning “an attractive man of an older age.”