Kunai were actually farm tools. Like little trowels. When the peasants rebelled against the government in Ancient Japan, a lot of them used what they had. Mostly farming tools. The edges could easily be sharped to points and used as knives. That's why Kunai are always portrayed with handles and shovel shaped blades.
They were used by peasants rebelling against the Japanese government, and yes, a lot of those peasants became "Ninja", which is where the whole ninja legend began, it is NOT a ninja-exclusive weapon, and makes more sense in the hands of a vagabond or thief than a ninja. Any well-trained ninja, like the ones portrayed on TV, would use a weapon of power and balance, like a Shuriken star that was specifically designed as one, before they would use a weapon of convenience, crafted of what was available, like a Kunai.
To be entirely truthful though, the idea that Ninja throw Shuriken stars or Kunai around all the time is fairly false. As previously stated, most Ninja began as peasants and farmers, and other rebels against the government. While some were ex-samurai or other trained persons, most were not martially-trained, and wouldn't have had the skill to use throwing weapons effectively. Back in ancient Japan, you would probably be more likely to see a Samurai use a Shuriken to cause their enemy to try to avoid it, shifting their focus away from the katana blade that was waiting to pierce their side, than you would be to see a Ninja using them.