First question I have is how much sun will these new plants have to live in. Makes a major difference. As for a shrub that is low maintenance, there are many. Oak Leaf Hydrangea is one I could heartily recommend. Is a native, has great "flowers" most of the summer, but likes some protection from midday sun. In PA, may not make as much difference. Withhazel, Carolina Allspice, Allegheny Serviceberry, dogwood (even the "twig" type), buddleia, and even redbud. Azaleas - there are some natives that are deciduous and could grow in your area as well as all the evergreen ones you can get at garden centers. Rhododendrons, are native and may grow, but they generally need some shade as well as Kalmia. As far as herbaceous, it would be hard to beat peonies (can be very long-lived), lilies ("bulbs" that are hardy), but don't like mole or voles playing with them. A new to me perenniel that I've grown to cherish is hellebore (very deer-resistant), but like shade. I love hostas, I have over 150 varieties, but they generally need shade, and I think deer like them. There are a number of native Viburnums. Carolina Silverbell is a stunning tree in flower and doesn't grow huge. Same with Sourwood. The bush magnolias (tulip and star) are good choices. Non-natives could be Japanese maples - there are hundreds of varieties to choose from. Trident Maple is a maple that doesn't get too big. I have a 40+ year old one that is maybe 20'-25' tall. I bought it to bonsai, but because of traveling, did not have time and just planted it in the yard. There are lots of choices in smaller or even dwarf forms of evergreen trees and hollies. Boxwoods and yew can be low-maintenance if you choose smaller varieties, however, boxwoods can die out in very cold temps.
Just about any fruit can be grafted onto dwarfing rootstock and yield a smaller to drastically smaller tree. I have one apple left that I grafted onto very dwarfing rootstock, 40+ years ago, that is maybe 6' tall. Or, you can control size by pruning - especially summer pruning. If you want to chance it a little, however, they seem to not be bothered by temps down to single digits and they bloom late (after probable frosts) is the Japanese Persimmon. If choose correctly, they self pollinate and some are not as tannic even when not as ripe.
Don't know about deer-damage on any of these.