One of the biggest changes in lending guidelines, was that appraisals can only be ordered by lenders through third party intermediaries, who randomly select appraisers from their registered pool. It has made the appraisal process theoretically more "impartial", where no individual or entity could influence the decision.
The flaw in this, is that no criteria or a system is set for selection of the appraisers, as long as they are willing to travel to the area where they are assigned a job. Often flawed appraisal reports are attributed to appraisers unfamiliar with the market area or in some cases just lack of work experience. What are the odds of an appraiser who knows the area well and has a decade of experience under his belt will come in with the same impartial opinion as someone new to the area and newly licensed? My gut says - highly unlikely.
Also important to note, that although for sale appraisals and refinance appraisals comparative approach is used to arrive at value, appraisers are more likely to come in at sale contract price, than appraisers doing a comparative valuation for a refinance.
Remedy? - I'd say firstly it's important to understand that the lender appraisal is NOT A PRICE INSPECTION! It carries no more weight than one individual's OPINION of what something is wort for single purpose of managing a lender's risk. Nothing more. It in no way devalues a property. Property's worth is determined by what potential buyers are willing to pay for it.
Second, most appraisers welcome VALID data. When facilitating appraisals we always make sure to supply the appraiser with recent sale comps, active sale comps, building data and floor plan. This does not guarantee that the appraiser will agree with the data, or won't go out and try to find more, yet it decreases the margin of valuation errors.
Third - though not applicable in case of a refinance, in case of a sale, it's not taboo to discuss the possibility of lower bank appraisal with a potential purchaser. Especially with the knowledge that it's been known to happen from time to time. If a buyer is property educated on the market, they will not only stick with the deal in the event of lower valuation, but will press their lender harder to correct it.
I hope this is helpful!