Refuse to buy it from them

The first shopkeeper they encounter might recognize it as an invaluable family heirloom, and refuse to buy it in those grounds. On the other hand, he might also recognize it as worthless, and offer only 2 copper for it. This might key players in to believing that they're meant to hold on to it, without giving away any of its power or true future value.

Give it a roleplaying purpose

Perhaps an innkeeper at a nearby town spots the sword on your player's belt, and makes it into a conversation piece. Perhaps she tells a story of the sword's long history, and the battles that Checkov's family fought with it, or perhaps she implies that the fact the sword was given away means that there's been an established trust between the players and this family, and that may open doors for them later in their journey. This option may key the party in to some idea that the sword has material importance, but it may be in a way they don't expect, and with the right group, this way could lead into some great experiences and development.

Let them know what it can do

Give your player's a story about how the sword was once a great weapon of dragon slaying, but it's magic has since been lost. Let them know that only a very difficult ritual could reactivate those powers and that it might take time and several levels before it can be re-activated. This can encourage them to pursue certain objectives, or lampshade a BBEG earlier than you may want, but it certainly ensures that the party keeps the weapon.

As a variant on the "let them sell it" answer, don't give it to them. Instead, have the farmer offer them some reward that he can supply because he's a farmer, like stabling for their horses, or other supplies that he generates on the farm.

However, make sure that the PCs know he has an old sword, with legends attached to it, and that one of those legends has unfulfilled clauses. If you play this right, the PCs will come back to the farm regularly, and the farmer will become a friend. So when they discover the sword they need is the one he has, all they have to do is ask him for it, although few other people know that it still exists, never mind where it is.

Make the merchant act suspiciously every time they try to sell the sword.

One could ask for a sacred expertise of the sword. Another can call it a replica and throw the PC out of the shop, claiming that he will chop heads if he ever see them again.

Or Merchant could see the inner value of the item and try to buy it extra cheap because if the party knew about this sword they would never show it to a simple merchant like him.

Or the merchant could be willing to pay any amount of gold for the sword. When the players try to bargain for the price, say "Deal!" the second they say the 1st price. And "Deal!!!" every time they try to raise the price.

If they still manage to sell it.. Just wait till they realize that was a mistake. And profit...

Give them a clear sign that the sword is more important than it seems. Some suggestions:

• When the magic-user of the party touches the sword, they feel something magic about it, but can't quite make out what it is.
• The next night, have the character carrying the sword have some vague prophetic dream about it.
• When the importance of the sword has nothing to do with magic, put some inscriptions on it which the farmer can't read (him being illiterate is not implausible) but which say something important to the more educated player characters. Have them roll every knowledge check in the rulebook. Most players will get the hint that you wouldn't bother having them roll when it wouldn't matter. The information you give them on success doesn't need to be helpful enough to be a spoiler and you can also put some information into what you say on a missed roll ("Due to your poor history knowledge you can not remember the name of the mighty hero who once used this sword or what existential threat to the universe it was which he slew with it").
• Have some NPCs act suspiciously interested in the sword. For example, you could have an NPC approach the PCs shortly after they acquired it who offers them some outrageous amount of money for it (how does he even know the PCs have it? Another mystery to solve for your players). But have that NPC insist that the trade happens in some suspiciously convoluted manner, like through dead drops (which end up being ambushes) or middle-men (which turn up dead).

Hint at the Future Use

Whatever the future use will be, somehow drop a hint that it will, against all odds, be useful later on. Like, if it's going to be traded to a distant relative of Chekhov, have a young relative see them with it in town and mention that he thinks his uncle in another town might be interested in it.