Is it too late to try and start a full collection? ...
Never too late, as long as you have deep pockets.
But if you are on a budget it's best to give up now. An Amiibo collection wouldn't be cheap at retail pricing, but when a significant number of characters sell for 4 or 5 times (or more) retail pricing your options will probably be limited.
Still disappointed I can't get a Villager Amiibo, or the whole set of Kirby-related figures.
This is true, collecting amiibo is possible but money will be a concern. I bought an Ike amiibo just recently from amazon for $30 alone. It's about double the cost and this Ike shipped from Japan since they are hard to find here. If I didn't have the extra money laying around I would have not of bought Ike in the first place. Honestly i'm happy to have Ike but this just might be the only time I will order through a third party for an amiibo.
In regards to retail (unless your getting a regular amiibo,) it is such a mess at times that getting a rare amiibo like Rosalina or even Ness basically requires magic, tricks, or the lords blessing to even have a chance to get them. On the bright side if you were able to defy the laws of our dimension by getting a chance to hold and buy a rare amiibo from retail it's a lot cheaper than buying from a third party. But the cost of getting... all of the amiibo's might not be worth the cheddar.
It's really not worth it to collect them all in my honest opinion. Personally I just advise(unless you have the lucky charms gold mine and resources to find and collect amiibo), getting the ones you like and especially if you can get them normally, by normally I mean not paying two times or more for one.