It's earliest usage was simply to refer to a bundle of sticks. The first written example of the word was in the 1300s.

Around the 1600s the word faggot was often used to refer to heretics who were burned at the stake unless they recanted their heresy. Those who recanted their heretical ways were made to wear an embroidered emblem of a figure of a faggot (bundle of sticks) on their sleeve.

Homosexuality has been associated with heresy since 1116, when the Henricians were condemned as heretics. From that point forward, the Church explicitly alleged that heretics practiced vir cum viris (man with man) and femina cum feminis (woman with woman). By 1209 Pope Innocent began the Crusade against the heretical and sodomitical Albigensians in France, and the burning of heretics (labeled as homosexuals).

Sometime around the late 1590s was when faggot is believed to be used as a pejorative against women. It's believed that during the 1600s when the term was used as a pejorative for heretics, that's actually when faggot could've been used for the first time to refer to homosexuals since many heretics were also accused of engaging in homosexuality.

The word faggot at this time was put towards the heretics due to the bundle of sticks they'd be burned over, but as well tied into the pejorative for women (think witches and homosexuals being feminised derogatorily).

That being said, the word "faggot" as meaning just a homosexual male was a creation of the US at around 1914.

So you are correct in the sense that faggot has been a pejorative for quite some time prior to it being solely for homosexuals, however, homosexuality and heresy were often closely associated during the era when faggot became a pejorative. It could be argued it was used against homosexuals since the 1600s.

Citations

The term faggot or fagot, meaning bundle of sticks, shows up around 1300 in English. It almost certainly came from Old French, possibly going back to Greek phakelos. Since those bundles of sticks were mainly used for fires, it's not surprising that the term came to mean burning sticks. Then there was that nasty business in medieval times where heretics were burned at the stake. Some later cites indicate heretics who repented and were spared a fiery death had to wear a picture of a faggot on their sleeve to show what might have been their fate. But no print evidence exists that homosexuals were referred to as faggots before the twentieth century, with the origin definitely in the U.S., not Britain.

As a last thought, a current notion holds that the Yiddish word faygeleh, "little bird," might have been the source, but lacks evidence other than the claim that the word was commonly used in Yiddish prior to WWII to indicate a homosexual. Source: Straight Dope Resources: Random House Historical Dictionary of American Slang, edited by J. E. Lighter, New York, 1994-1997. Oxford English Dictionary Rictor Norton, A History of Homophobia, "4 Gay Heretics and Witches" 15 April 2002, updated 18 February 2011