About a month ago
@saundersbp and I went
We flew to Ancona airport and got a ride to Loreto from our booking.com host.
This is where we stayed, reasonably priced when you split the cost between two people and is a short walk away from the basilica and shops. For €50 the host waited for us at the airport, gave us a tour of the area and dropped us off at our accommodation with a couple of recommendations on where to eat. You could self cater if you like, the apartment had everything you'd need.
As for getting around the area, the buses are unreliable - or rather the information about them is. We had a printout of bus times from the tourist information centre, google maps, and the digital timetable at the bus stop. All 3 said different things, and when a bus turned up at what seemed to be the right time (according to the digital bus stop) the driver said no no, the one you want is in a few minutes - which wasn't on any of the info we had. But sure enough in a few minutes our bus arrived.
You have to buy the bus tickets in advance from local newsagents/bars, not a big problem but just something to be aware of.
Most of the factories are not in Castelfidardo. There are a handful of showrooms in the town centre, but most are in the general area between Loreto and Castelfidardo. We tried walking to the Bugari factory from the closest bus stop but it was too dangerous of a road. High speed traffic with no pavement to walk on, which seemed to be the norm once you got out of any of the town centres! Luckily for me the factories were rolling the red carpet out for Ben and collected us in their car, but I don't think they would've normally done that.
If I were to go again without the benefit of of the exceptional hospitality from the factories and accommodation hosts I would definitely just rent a car and save some headache. The issue then becomes finding places to park.
I would highly recommend staying in Loreto rather than Castelfidardo. It's bigger, has more shops, places to eat, things to see, etc.
P.S. the food is fantastic!