Showing posts with label QA-Review AVM. Show all posts
Showing posts with label QA-Review AVM. Show all posts

Monday, October 26, 2020

How to Conduct a Successful Re-assessment – Preemptively

Many property re-assessments produce sub-par results or even fail miserably due to a straightforward fact: jurisdictions often take the full-scale plunge into the venture without conducting meaningful pilots, leading to proper definitions of scale and scope of the actual event. Therefore, running a well-thought-out pilot could save a ton of money and agony (in terms of public and political embarrassment, etc.) down the road.


1. Ideally, Conduct Residential and Commercial Re-assessments concurrently. When they are run together, local governments are empowered to shift tax burdens across property groups (depending on the impact study). If it is statutorily required, the taxpayer watch groups must fight the statute to decouple them, thus making the re-assessment a genuinely transparent, as well as a fair and equitable exercise. If it is run concurrently, the watch group must hire an independent consultant to review the impact study, both inter (across property groups) and intra (within the group). Should they find any inconsistency, they must share the results with the local media.


2.  Hire an Econometric consulting firm to run a Pilot. Running a meaningful pilot is one area where private and public sectors tend to part ways. For example, instead of rushing into a full-scale (and expensive) marketing campaign, private companies tend to run a meaningful pilot (i.e., proper sampling, etc.) first, leading to the primary campaign, assuming, of course, pilot results exceed expectations; just meeting expectations could force the project back into the mix of alternatives). Though the idea of pilot projects is not common in local governments, they must get into the practice of running pilots to avoid having to spend too much money at the back-end on damage control. Since a well-constructed and adequately run pilot represents the main event, a well-known econometric consulting firm must do justice to the pilot, paving the way for a meaningful and significant pilot and a reliable impact study.    


3. Recollect the Exterior Data for the Pilot Project as if it were the Main Event. Before publishing the data collection manual, the consulting firm must undertake a local market significance study, thus zeroing in on the variables that significantly impact valuations in that particular market. Then, with the assistance of the consulting firm (e.g., arriving at the actual sample, variable types, extent, and use of technology, etc.), the assessing staff must recollect the exterior data on the pilot. In constructing the sample, it's prudent to ignore all incomplete and on-going physical changes. Similarly, the interior data collection is virtually meaningless for re-assessment as they represent mostly lifestyle fixtures/personal properties, not real properties. While significant interior renovations and improvements must be captured and reflected via the "Overall Condition" variable, new indoor pools, porches, etc. should be separately coded to ease valuation (only if they show up in the study as significant market variables). The data collection process must be thoroughly documented so that the process could be precisely duplicated during the main event.


4. Publish the Pilot Results, emphasizing the Potential Tax Impact. Considering this is not the actual re-assessment, the results could be published immediately, with a series of outreach seminars to educate taxpayers on the potential impact of the future re-assessment. Even the taxpayers facing tax increases would be less hostile at this point as they would be allowed a significant voice in reshaping the outcome. If the residential and commercial pilots are run concurrently, watch groups must carefully scrutinize the study, ensuring that the tax burdens are not being irrationally shifted from one group to another, especially "inter," meaning from the commercial to the residential. Of course, they must also study the equity within the group. Of course, that is the advantage of a meaningful front-ended pilot, providing a platform for all brainstorming before the fact.  


5. Jurisdictions with Unfair Statutory Limitations must work on Removing the Statute before Undertaking any Major Re-assessment. Hypothetically, if the state mandates that the county must reimburse its taxing districts (e.g., towns) the amount that is refunded to homeowners due to inaccurate property assessments, it would be prudent for the county administration to work with the state to remove this unfair mandate, or at least reduce the burden to a manageable annual limit graduating to a total phase-out) before embarking on any significant re-assessment. Should this legislative effort fail, the county should seriously consider a decentralized assessment system without taking on monumental unwarranted liability. In this example, under the decentralized system, towns would be responsible for their assessments. Simultaneously, the county would continue to provide technical assistance, thus relieving the county of any potential refund liability.


Again, a front-ended pilot would do immense good before the full-scale plunge.


-Sid Som
homequant@gmail.com


Thursday, November 7, 2019

9 Issues that make an AVM Inefficient, often Ineffective

The difference between a smart modeler and an average modeler is that the former quickly figures out and intelligently avoids the (valuation) modeling that tends to make the process inefficient, if not ineffective. That is why establishing a quality control or a modeling review process is essential to avoid having to deal with significant damage control down the line. A handful of such poor practices may raise serious concerns for the quality control reviewers. Here are some of those poor practices.

1. Time Adjustment: Some practitioners use the number of months since sale (NMSS) as an independent variable to ascertain the rate of growth (+/-) in the targeted price level, paving the way for the time coefficient to adjust the modeling sale prices, which then serves as the dependent variable in the regression equation. While this is acceptable as a lead-up regression to generate the primary dependent variable, it is unacceptable as an independent variable going into the regression equation. The reason is simple: since NMSS would be missing in the unsold population (the model would eventually be applied to), resulting in the application to fail. 

2. Sales GIS: Testing the Sales GIS representativeness is not an easy proposition, forcing many practitioners to skip this sampling test. Sales GIS is often a function of the market dynamics, deviating from the Population GIS. Therefore, an untested Sales GIS paves the way for an inefficient AVM. That is why many practitioners tend to use "fixed neighborhoods" in the modeling process as they are more stable and well-accepted, meaning they do not succumb to the short-term market swings and are generally liquid enough to help test the representativeness of the modeling sample.

3. Chasing Trophies: An AVM is not meant for the entire population. Trying to achieve a maximum of two sigma solution (95%) is more meaningful than the whole population, leaving out the admixture of the utterly unattainable 5%, including the limited number of trophy properties and large mansions. The reason is apparent: The paucity of such sales. Therefore, the smart modelers remove them from the modeling spectrum altogether, i.e., they do not let any of those sales enter the modeling sample, nor do they apply the model onto that subset.

4. Chasing Tiny Bungalows: This is the flip-side of dealing with the trophy properties. There is a tiny waterfront (in fact, many of them are prime oceanfront, e.g., Point Lookout, Atlantic shore Long Island, NY) bungalows all around the country, but this subset represents the land values primarily. Once they are sold, they generally take a completely different form, often as multifamily properties with re-zoning. Therefore, as long as the existing improvement is sound, they must be hand-worked by the appraisers. Letting them into the modeling process would be a red flag for the quality control reviewers.

5. Combining 2, 3, and 4-Family with SFRs: Considering this sub-class of residential properties is mostly the income-producing, the smart modelers know it's imprudent to a group and model them the single-family residences (SFR). New modelers often mistake combining them with SFRS as they are part of the same tax class in individual states or sharing the same mortgage category. Of course, the properties within this sub-group are mostly transacted using comparable sales to be market modeled, but as an independent and mutually exclusive (of SFRs) group. Mother-and-Daughter, a relatively common form of set up in big cities, is not a technical 2-family to be modeled with the SFRs.

6. Synthetic Variables: Many modelers who come from the non-quantitative background become obsessed with better modeling stats, allowing irrational or unexplainable synthetic variables like (X * Y) ^ Z into the equation. Grated, such variables may enhance the model's favorable stats but reduces its explainability and decomposability, and therefore, it's overall utility! On the other hand, the modelers with sound quantitative background realize from the get-go that the use of such variables is nothing but sowing the seeds for inefficient modeling, knowing very well they could never explain the underlying market economics.

7. Untested Models: As explained in the previous chapter, it's always a prudent practice to test the draft model on to a mutually exclusive hold-out sample before being applied on to the population. The hold-out sample test must produce very similar results, both before and after the outliers, as in the draft model, ensuring that the real model (not an interim version) is being applied. Caution: Since the hold-out sample is part of the original sales sample and therefore comprises sale dates, it will NOT detect the NMISS issue (# 1 above). Again, this is a critical step, the absence of which is tantamount to adventuresome modeling and could be costly at the end.

8. Sales Complex: Any part of the sales complex, directly or indirectly (e.g., ASP/SF, etc.), must not be used as independent variables in a regression equation as it violates the basic assumption of multiple regression analysis. NMISS is a classic case of such violation of the standard modeling practice. To reiterate, when any part of the sales complex is introduced on the independent side of the regression equation, it could not be applied to the unsold population as it would lack the sales attributes. Of course, this is one of the first rule violations any qualified quality control reviewer would look for.

9. Lack of Value Optimization: Since the final quality of the vast majority of the CAMA models is primarily error-based (usually the last model's Coefficient of Dispersion/COD, Price Related Differentials/PRD, etc.), there is hardly any practice of optimizing the final values from the regression models. In other words, those models are not subjected to the real test of optimality of the solution; for example, the CAMA modeler can hardly answer if their model COD of 8 is better than the model COD of 10. In individual events, the COD of 8 could be a post-optimal solution despite being a better error stat, while the COD of 10 could have been the optimal solution. Therefore, the final regression values need to be optimized via Linear/Non-linear Programming.

To conclude, modelers must remember to get into the habit of producing industry-standard models, without getting carried away by the quality stats. Case in point: A model with a conforming COD of 12 could receive a much better quality control score than its counterpart with a COD of 9.  

-Sid Som, MBA, MIM
President, Homequant, Inc.