A large space that provides room to run and protection from predators is important for goats. A minimum of a 16'x16' space is recommended for a pair of goats, although something larger would be much better as they are active and curious animals. Fencing should be free from rusty wire and fool-proof: goats are escape artists!
A shelter that fits all of the animals in the pen comfortably and protects them from wind, cold, and heat is also necessary. Something as simple as a wooden doghouse that keeps the winter chill off their back can be the difference between a goat with pneumonia and a healthy goat.
Check in with your local extension office to find out about local poisonous weeds, and make sure they are removed from your property. Foxtails can also cause serious problems for grazing goats when they are dry and should not be near the goat's pen.