 |
| (Click on the image to enlarge) |
US Data/Portfolios Only
Our Free Home, Condo & Auto Valuations
Showing posts with label Automated Valuation Model. Show all posts
Showing posts with label Automated Valuation Model. Show all posts
Thursday, October 29, 2020
Subscription-based Data Analysis and Modeling
Our Free Home, Condo & Auto Valuations
Wednesday, October 28, 2020
How to use AVM to Discover Over-assessed Parcels on Tax Roll
Comparing independently developed Automated Valuation Model (AVM) Values to County Market Values (CMV) will point to the areas of failure, meaning over and under-valued assessments on the Tax Roll. Often, the higher value properties are under-assessed, while the lower value properties are over-assessed. If the comparison of "AVM to CMV" points in that direction, the Property Tax Appeals Consultants ("consultants") must work up a small sample, using comps, to further authenticate the discovery. If the comps sample validates the discovery, consultants must pay special attention to that over-valued/ over-assessed population segment.
In choosing AVM Vendors, consultants must ensure that those AVM Values are developed specifically for the Tax Status Date. If the Valuation date (or the Tax Status Date, as the case may be) is 1-1-2019, but the AVM Values were developed in June 2018, those values would produce a flawed picture when compared with the County Values. Therefore, it is advisable to work with AVM Vendors that develop custom or specialized models for the Appeals industry, per se.
Many AVM Vendors also sell Comps Reports. However, the Appeals Consultants must be careful in working with the specialized AVM Vendors who additionally tie their AVMs to the Comps production. In other words, the specialized AVM Vendors who use the model coefficients to adjust their comps via the Comps Adjustment Matrix do not necessarily produce the most optimal comp reports as AVM (top-down) and comps reports (bottom-up) are diametrically opposite solutions. If a consultant is looking for a long-term AVM vendor, this is always worth asking, meaning if they tend to tie their comps (reports) to the model coefficients.
In the due diligence course, consultants may ask for a sample Adjustment Matrix for the comps production. The sample itself will say a lot about the quality of their valuation process. For example, if the Comps Adjustment Matrix shows a 'Lot SF' coefficient of .10 (10 cents per Lot SF, assuming it's transferred from the regression model producing the AVM model values), it would be difficult to explain such adjustments to the clients looking for self-explained comps. It would be a clear indication that the vendor is working with totally unqualified "make-shift" modelers.
While no AVM Vendor would be forthcoming to show their AVM models, they might share a sample Comps Adjustment Matrix. It could be telling!
Many Consultants use free or low-cost home valuation sites to work up samples. Unlike the free brokerage sites, some of those sites are generally self-directed, allowing them to arrive at their value conclusions. A few of them even comprise a host of advanced features like subject simulation, comps selection, quantitative adjustments, distance matrix, time adjustments, flexible valuation dates, multiple ranking methods, interactive spatial interface, comps grid, value analysis, and an all-inclusive report, to name a few. Some newer sites are mobile-friendly (so no additional Apps are needed) and strictly top-down, providing "quick look" subject valuations. Of course, those who subscribe to the local MLS have access to the comps solutions as well.
In any case, having access to a custom or specialized AVM will help consultants to isolate the meaty cases from the Roll, thus narrowing the competition down. It makes no economic sense to go after the cases which are valued at or below the market.
-Sid Som
homequant@gmail.com
Tuesday, October 27, 2020
AVM is a Market Solution, Comparable Sales Analysis isn't (Part 2 of 2)
If a subject and a sales population are provided to a group of concerned parties – from an Assessor to a Bank Appraiser to a Listing Agent offering buyback guarantee to a traditional Listing Agent to a Buyer's Agent to an Appeals Consultant – one would be unpleasantly surprised by the outcome.
They will pick different comps based on their professional requirements and objectives, leading to different, often very conflicting valuations. For instance, Assessors may not have the taxpayers' best interest at heart as they have to meet budgetary requirements, paving the way for counterparties like Appeals consultants. A Listing Agent looking to get an "exclusive" may not do well with a set of middle-of-the-road comps that a Buyer's Agent might be interested in. In other words, the selection of comps is a function of the hat the party wears, making the entire process highly subjective. AVM, on the other hand, is a reasonably scientific exercise. All variables interact with one another in an econometric equation and produce the resulting values. Therefore, all other factors remaining constant, two identical homes will have equal values – but not so in the world of the comparable sales analysis (aka, comp sales) as it is very party-specific.
Once the sales pool that closely represents the subject is scored correctly and quantitatively adjusted, it becomes comps. Generally, the five best comps are then selected to value a subject. Valuers tend to use one of the three standard methods – distance, least adjustments, and sales recency – to narrow their choices down to the five contributing comps.
In this analysis, the subject home attributes are Bldg SF=3,250, Lot SF=17,400, and Bldg Age=26. An optimal pool of 10 comps was algorithmically produced from an extensive sales population to demonstrate how subjectivity plays a vital role in this valuation process, removing the lowest ($308,770) and the highest value ($422,175) comps in each approach.
The above table represents the distance method, meaning the five closest (to the subject) comps were considered to be the best comps, producing a value range of $344,820 to $414,940, with a probable subject value of $388,775. Since least adjustments and recency of sales were ignored here, obviously, several comps needing large adjustments or of older originations managed to creep in, making the process sub-optimal.
The above table represents the least adjustment method, meaning the comps that required the least adjustments were considered the best comps. The least adjustment is nothing but a balancing act. In other words, larger lots are compensated in value by smaller building sizes, and lesser time adjustments are proxying for older homes, etc. For example, the second least adjusted comp (# 6) with a much smaller lot was corrected by the larger and older building. It also sacrificed one of the closest (# 8) comps. This method produced a lower subject value of $371,150.
The above table represents the sales recency method, meaning the most recent five comps (in terms of sale dates) are the best ones. This is where the lowest and the highest value comps showed up on the initial line-up, hence substituted with the ones waiting in line. Though this method produced the most compact value range (upper bound was compacted down), it produced the lowest subject value of $360,340.
Therefore, if this comp sales analysis were to be used to cater to the target as mentioned earlier audience, this is how the game would be played out:
1. Assessors and Listing Agents (traditional) will be given the "distance" value (highest value).
2. Bank Appraisers and Listing Agents (buyback) will be given the "least adjustment" value (middle-of-the-road value).
3. Appeals Consultants and Buyer's Agents will be given the "sales recency" value (lowest value).
How to Reduce Subjectivity in Comp Sales
1. Apply meaningful selection, scoring/ranking, and adjustments to the sales population;
2. Build an AVM and insist on two AVM values (4th and 5th) on comps line-up;
3. Verify all comps spatially, ensuring they all come from the same or, at least, compatible neighborhoods;
4. Apply time adjustments in line with the local market (using national figures or adjustments could distort results);
5. Pay attention to valuation dates as 01-01-19 and 08-16-19 are different, often requiring additional adjustments;
6. While using sales recency, contract dates are preferred to closing dates (despite the industry norm);
7. If one is not allowed to use AVM values, one must show the AVM values below the comps grid with detailed value analysis;
8. If the sales population is large, a representative sample might be extracted from the most recent arms-length sales; and
9. If the subject population is large, automate the process with batching technology (batch comps).
AVM is a Market Solution, Comparable Sales Analysis isn't (Part 1 of 2)
 |
| (Click on the image to enlarge) |
In developing the analysis, the same sales population – derived from a single Zip Code – has been used across all three graphs (of course, one may use other fixed locations like Census Tract, School District, etc.). Considering all sales originated in the same Zip, it helped minimize the impact of location (of course, one can never make location totally irrelevant as each block has a different appeal).
The above graph shows the noisy relationship between the uncorrected (raw) Sale Price and Bldg Size (Heated Living Area). The reason is straightforward: Each sale is directly related to a buyer's judgment, causing a high level of subjectivity; for instance, the buyers are paying between $100K and $250K for a 1,500 SF home. While the investors would target the lower end of the range, the informed buyers would be in the middle, and the uninformed buyers (someone bent on buying a pink house!) would succumb to the higher end of the range. Therefore, the R-squared is extremely low (0.189), thus explaining very little of the variations in sale prices.
The Regression Value-1 graph proves that even a rudimentary regression model (with only three independent variables - Land SF, Bldg SF, and Bldg Age) can produce a decent market solution. The fit is significantly tighter, especially at the long end of the curve. The R-squared jumps from 0.19 to 0.91, accounting for 90% of the variations in sale prices. But this model has bi-modal issues between 1,000 and 2,200 SF as the regression values are forked. It's important to note that such stacked values must be investigated for the underlying reasons. One of the simple ways to identify the issue is to scatter the normalized regression values against the other independent variables and look for possible explanations.
The above investigation guides to the solution. As the normalized regression values from the first model were scattered against the Bldg Age variable (above graph), it was evident that many buyers were paying a premium for the younger homes, causing the stack. A sizeable portion of those buyers was willing to pay over $130/SF for the younger homes, while very few offered such a premium for the older stock. More precisely, none paid over $160/SF for the older stock.
So the Bldg Age variable had to be transformed from continuous to binary (younger homes vs. the rest). The regression model's re-run with the transformed Bldg Age produced the above (Regression Value-2) graph. Consequently, the value fork has disappeared, translating to a much tighter fit, with improved R-squared, lower intercept, and a steeper slope approaching 45 degrees.
Stay safe!
-Sid Som
homequant@gmail.com
Benchmark AVM and CAMA Modeling Service
 |
| (Click on the image to enlarge) |
US Tax Jurisdictions Only
Our Free Home, Condo & Auto Valuations
Wednesday, October 21, 2020
Automated Valuation Modeling (AVM) for SFR Rental Portfolios
 |
| (Click on the image to enlarge) |
US Tax Jurisdictions Only
Our Free Home, Condo & Auto Valuations
Tuesday, October 20, 2020
Promotional AVM & CAMA Modeling Services
 |
| (Click on the image to enlarge) |
Our Free Home, Condo & Auto Valuation Sites
Thursday, October 1, 2020
Promotional Sales Ratio Study
 |
| (Click on the image to enlarge) |
US Data/Portfolios Only
Our Free Home, Condo & Auto Valuations
Thursday, September 10, 2020
These 9 Enhancements will make most Assessment Rolls Fair and Equitable
Despite statistical modeling, the vast majority of assessment rolls remain moderate to highly regressive, meaning the middle-class homeowners often subsidize the upscale homes. In other words, the homes in the lower-to-mid value range tend to be over-assessed while the higher value segment gets a significant break. The use of the antiquated CAMA modeling cannot fix that problem in major jurisdictions with highly complex housing stock, but setting and adhering to the following issues would take those jurisdictions in the right direction, leading to more equitable tax rolls.
1. Backward-bending Status/Valuation Date: Since the Taxable Status Date (or the Valuation date) is often a futuristic date, the available market information and data tend to be quite inadequate to develop proper predictive (mass appraisal) models that, in turn, generate the assessment roll. Case in point: Many taxing jurisdictions utilizing the CAMA (mass appraisal) modeling generally build their models in August/September with the available data, targeting the next January as the status date, creating an enormous predictive (data) gap. A vast majority of those modelable sales would have been contracted in the first and early second quarter of the year (the gap is direr!). Therefore, the status/valuation date must be set back in line with the availability of the modelable data, to avoid having to produce (or gamble with) a futuristic roll.
2. Short-term Sales: When the holding period of the property is less than two years, those sale prices must be anchored as market values, thus forcing the potential gamers (institutions, flippers, etc.) to pay higher taxes. The point is that the housing market's rapid institutionalization needs to be evaluated separately, without having to redistribute their share (of the burden) on to the rest. Therefore, assessors must aggressively lobby to remove the application of any state/charter restrictions (e.g., annual growth capped at 5%, etc.) from short-term sales, so they stand on their own at the market level, without any pseudo protection. Exclusion: non-arms length sales (inter-company, distressed sales, etc.).
3. AVM/CAMA Application: Since Automated Valuation Modeling (AVM is a top-down econometric solution encompassing the vast majority of residential properties on the roll) is inherently more equitable, it must be applied to the rest of the tax roll, including the non-arms length from #1. An independently developed challenger model should point to the outliers at the population level, requiring them to be separately hand-worked (using the analysts' traditional comparable sales approach). Depending on each roll's complexity, these models generally work well between the 10th and 90th percentile of the value curve, so the rest should be scrutinized and perhaps hand-worked. Unfortunately, failing to pay proper attention to the aforesaid outer bounds often degrade the entire roll. The point is, AVM/CAMA should be applied where it is applicable.
4. Unique Properties: Again, due to the paucity of arms-length sales, AVM/CAMA values are not meant for this group (trophies, mansions, large waterfronts, tiny oceanfront bungalows, etc.), so they must be hand-worked by the assessing staff. The sale of unique or vastly atypical properties must not enter the modeling (AVM) sales sample. Right at the outset of modeling, separation of such categories must be discussed and isolated from the modeling spectrum altogether. Such decisions help enhance the marketability of the roll. Still, when the modelers arrogantly apply models to these strata, the values are generally indefensible, forcing taxpayers and other external agencies, including media, to question the entire roll value.
5. 2-to-4 Family Properties: Although assessment and mortgage statues generally lump these properties with the single-family residences (SFR), they must not co-share the SFR roll, considering these are inherently income-producing properties. Therefore, they must be modeled, valued, and taxed as a separate group or as a sub-group (small cap) under the multifamily group. Depending on the liquidity of this group, they could even be market modeled standalone or hand-worked. If the market approach, rather than the income approach, is used to value them, that approach must not change from year to year as that could introduce significant inequity.
6. Outsource AVM: It's much cheaper to outsource residential AVMs to a financial consulting/research firm (but not an appraisal or CAMA outfit) than maintaining a dedicated group of in-house modelers. It will also be more effective as those firms are non-political, and as such, their feet would be held to the fire for proper performance following the performance metrics contained in the contract. For example, one such metric could be the level of appeals meaning if the appeals go up due to flawed assessments, they could be forced to make the necessary corrections promptly without any political interventions or cover-ups. Conversely, partisan politics plays a significant role when the values are internally generated, especially when the Assessor is not an independently elected official.
7. Manpower Planning: Larger jurisdictions must learn from the private (FedEx, UPS, Amazon, etc.) how to use the seasonal help. For example, the seasonal help could be used to process exemptions, appeals applications, income/expense statements, etc. They can be used to perform field operations and inspections too. Alternatively, counties/taxing jurisdictions may maintain a "Central Pool" agency to cater to the different agencies based on their seasonal workforce requirements. Maintaining full-time personnel to handle seasonal activities is often the case – is an expensive and wasteful proposition, returning minimal value to the taxpayers. Even when an internal modeling group is maintained, it must be hired and kept outside of the civil service/union system, allowing navigational adjustments.
8. Practice Assessment: Instead of having an autonomous appeal or review department, it's better to have it under the Assessment umbrella. Having a separate government agency or department is inherently political and is not necessarily in the taxpayers' best interest. Often, the focus of such autonomous appeals/review agencies is to hire appraisers, rather than trained assessors, which does not promote equity of the roll as they generally lack the understanding of assessment equity. Moreover, such autonomous political agencies quickly learn that the workforce's growth solidifies the ground under their feet, thus gradually distancing themselves more and more from the Assessor's office and growing into a powerful political body. None of this helps improve the quality of the assessment roll.
9. Commercial Assessment: To promote equity of the commercial segment of the tax roll, departments must practice assessment, not an appraisal. Therefore, it's prudent to collect a sizeable Income and Expense (I/E) sample to develop meaningful local metrics for the commercial properties. These I/Es could also be used to formulate use-based AVMs to process the homogeneous properties like 4-to-10 family rental properties, small office complexes, mixed-use properties, public storages, smaller warehouses, and industrial properties, etc., in turn, creating more equity across such denominations. This approach would free up assessors to devote more time to the complex commercial properties where modeling is futile or unnecessary. Commercial appraisers must also be trained and groomed to transition from an all-appraisal mindset to an assessment equity-oriented thinking.
Flood Zone Insurance: Last but not least, it's essential to enforce the flood zone insurance. Homeowners inside the designated flood zones must be required to carry flood zone insurances. Should the situation arise, insurance companies would be on the hooks, not the rest of the taxpayer population. Of course, an annual credit could be offered to offset a minimum policy deductible.
Again, having an all-encompassing residential AVM does not make the roll fair and equitable; it must be properly sourced and targeted. Likewise, training and promoting a group of unqualified employees as AVM/CAMA technicians do real injustice to the roll. Finally, use-based Income models off of actual I/E data could promote equity across commercial property groups' homogeneous stretch.
-Sid Som, MBA, MIM
President, Homequant, Inc.
homequant@gmail.com
Sunday, September 6, 2020
9 Issues that make an AVM Inefficient, often Ineffective
The difference between a smart modeler and an average modeler is that the former quickly figures out and intelligently avoids the (valuation) modeling that tends to make the process inefficient, if not ineffective. That is why establishing a quality control or a modeling review process is essential to avoid having to deal with significant damage control down the line. A handful of such poor practices may raise serious concerns for the quality control reviewers. Here are some of those poor practices.
1. Time Adjustment: Some practitioners use the number of months since sale (NMSS) as an independent variable to ascertain the rate of growth (+/-) in the targeted price level, paving the way for the time coefficient to adjust the modeling sale prices, which then serves as the dependent variable in the regression equation. While this is acceptable as a lead-up regression to generate the primary dependent variable, it is unacceptable as an independent variable going into the regression equation. The reason is simple: since NMSS would be missing in the unsold population (the model would eventually be applied to), resulting in the application to fail.
2. Sales GIS: Testing the Sales GIS representativeness is not an easy proposition, forcing many practitioners to skip this sampling test. Sales GIS is often a function of the market dynamics, deviating from the Population GIS. Therefore, an untested Sales GIS paves the way for an inefficient AVM. That is why many practitioners tend to use "fixed neighborhoods" in the modeling process as they are more stable and well-accepted, meaning they do not succumb to the short-term market swings and are generally liquid enough to help test the representativeness of the modeling sample.
3. Chasing Trophies: An AVM is not meant for the entire population. Trying to achieve a maximum of two sigma solution (95%) is more meaningful than the whole population, leaving out the admixture of the utterly unattainable 5%, including the limited number of trophy properties and large mansions. The reason is apparent: The paucity of such sales. Therefore, the smart modelers remove them from the modeling spectrum altogether, i.e., they do not let any of those sales enter the modeling sample, nor do they apply the model onto that subset.
4. Chasing Tiny Bungalows: This is the flip-side of dealing with the trophy properties. There is a tiny waterfront (in fact, many of them are prime oceanfront, e.g., Point Lookout, Atlantic shore Long Island, NY) bungalows all around the country, but this subset represents the land values primarily. Once they are sold, they generally take a completely different form, often as multifamily properties with re-zoning. Therefore, as long as the existing improvement is sound, they must be hand-worked by the appraisers. Letting them into the modeling process would be a red flag for the quality control reviewers.
5. Combining 2, 3, and 4-Family with SFRs: Considering this sub-class of residential properties is mostly the income-producing, the smart modelers know it's imprudent to group and model them with the single-family residences (SFR). New modelers often make the mistake of combining them with SFRS as they are part of the same tax class in individual states or sharing the same mortgage category. Of course, the properties within this sub-group are mostly transacted using comparable sales to be market modeled, but as an independent and mutually exclusive (of SFRs) group. Mother-and-Daughter, a relatively common form of set up in big cities, is not a technical 2-family to be modeled with the SFRs.
6. Synthetic Variables: Many modelers who come from the non-quantitative background become obsessed with better modeling stats, allowing irrational or unexplainable synthetic variables like (X * Y) ^ Z into the equation. Grated, such variables may enhance the model's favorable stats but reduces its explainability and decomposability, and therefore, it's overall utility! On the other hand, the modelers with sound quantitative background realize from the get-go that the use of such variables is nothing but sowing the seeds for inefficient modeling, knowing very well they could never explain the underlying market economics.
7. Untested Models: It's always a prudent practice to test the draft model on to a mutually exclusive hold-out sample before being applied on to the population. The hold-out sample test must produce very similar results, both before and after the outliers, as in the draft model, ensuring that the real model (not an interim version) is being applied. Caution: Since the hold-out sample is part of the original sales sample and therefore comprises sale dates, it will NOT detect the NMISS issue (# 1 above). Again, this is a critical step, the absence of which is tantamount to adventuresome modeling and could be costly at the end.
8. Sales Complex: Any part of the sales complex, directly or indirectly (e.g., ASP/SF, etc.), must not be used as independent variables in a regression equation as it violates the basic assumption of multiple regression analysis. NMISS is a classic case of such violation of the standard modeling practice. To reiterate, when any part of the sales complex is introduced on the independent side of the regression equation, it could not be applied to the unsold population as it would lack the sales attributes. Of course, this is one of the first rule violations any qualified quality control reviewer would look for.
9. Lack of Value Optimization: Since the final quality of the vast majority of the CAMA models is primarily error-based (usually the last model's Coefficient of Dispersion/COD, Price Related Differentials/PRD, etc.), there is hardly any practice of optimizing the final values from the regression models. In other words, those models are not subjected to the real test of optimality of the solution; for example, the CAMA modeler can hardly answer if their model COD of 8 is better than the model COD of 10. In individual events, the COD of 8 could be a post-optimal solution despite being a better error stat, while the COD of 10 could have been the optimal solution. Therefore, the final regression values need to be optimized via Linear/Non-linear Programming.
To conclude, modelers must remember to get into the habit of producing industry-standard models, without getting carried away by the quality stats. Case in point: A model with a conforming COD of 12 could receive a much better quality control score than its counterpart with a COD of 9.
-Sid Som, MBA, MIM
homequant@gmail.com
How to Build a Better Automated Valuation Model (AVM)
Most residential AVMs are market models. Therefore, a representative sales sample is used to develop the model and then applied to the unsold population to generate model values for all unsold properties (assuming non-missing data, etc.). While it sounds like a simple process, it does require significant econometric knowledge to develop such a model. A smart modeler figures out how to take the adventure out of the modeling process and standardize it, thus stabilizing values from year to year. For example, once the process is standardized, leading to efficient values, smart modelers continue to introduce new sales but not new data variables, and by doing so, they take the two most essential features of valuation modeling – explainability and decomposability – out of the environment. Therefore, to standardize the modeling process and bring stability in values, the following sequential steps are needed:
1. Sales Sample: Assuming it's an "Application" AVM (meaning the model will be applied on to the population the sample is derived from), the modeler starts with a representative sales sample. While testing the sampling properties, all three variable categories – continuous (Bldg SF, Lot SF, Age, etc.), categorical (Bldg Style, Exterior Wall, Grade, Condition, etc.) and fixed location (Town, School District, Assessing District, etc.) – are meaningfully evaluated. A good representative sample is a must, lacking which the model would be inaccurate, producing faulty values. That is why smart modelers consider this as a make or break step.
2. Time Adjustment: Depending on the sales liquidity, a market AVM requires 12 to 24 months' worth of arms-length sales. Since price levels go up and down, monthly time adjustments are needed. When the time series is extended (24+ months), quarterly adjustments are more useful as they are smoother and more market-efficient (reduces inconsistencies arising from using the "closing" dates rather than the contract dates). Unlike sales analysis, time in AVM is a surface correction, so it's better done at the "outer" level; for instance, while modeling a county, it's better to keep time adjustments at the county level, without drilling down to the sub-markets. Also, the price-related (FHA, Jumbo, etc.) corrections must be avoided.
3. Hold-out Sample: Once the sales are time adjusted, the sales sample must be split up between the modeling sample (e.g., 80%) and hold-out sample (20%). Both sub-samples – modeling and hold-out – must have very similar attributes to the original sales sample representing the population. It's good to use one of the software-provided sampling procedures that help reduce the judgment. The model is developed with the modeling sample and then tested on to the hold-out. While the results will not be exact, they must be similar (very close).
4. Multi-stage Regression: Since the sample comprises three different types of variables, it's prudent to develop a three-stage regression model, piggy-backing the output from the prior stage. Considering that the contributions of each variable are generally non-linear, the log-linear model is more effective. If the dataset is comprehensive with many categorical variables, a correlation matrix is needed to determine any multi-collinearity (if specific variables are highly correlated), which often leads to a reduced number of variables. Conversely, if the number of variables is limited, the t-stat is generally an excellent metric to control the variables' significance levels. If a variable's t-stat is less than 2, it is typically non-contributing.
5. Multi-cycle Regression: To make the model efficient, it is a practice to develop it in three cycles. The sales ratios (1st Cycle AVM values to Adjusted Sale Prices) from the first cycle will help define and remove outliers. Then, the outlier-free sample will lead to cycle two at the end of the first cycle, generating pre-residual AVM values and ratios. The end of cycle two helps make residual corrections, which is a very time-taking and iterative process. Once all of the residual revisions are finalized, the third and final cycle is run, which generates the model and produces the final values. Since a smart modeler knows how to systematically and methodically develop a model, it's generally far more efficient than those created ad hoc or in one single step.
6. Residual Analysis: At the end of the second cycle, the residuals need to be worked on. The fact that some sales ratios are clustered around 70 while some are around 140 does not mean the AVM values are wrong or inefficient because the AVM values are being compared with sales, which, individually, are all judgment calls. For instance, a prospective homebuyer bent on purchasing a pink house would in all likelihood overpay, while an SFR Rental company or an aggressive flipper or investor would buy a group of properties, perhaps paying way below the market (although some of those properties would be coded as arms-length), etc. In other words, the model is essentially fixing those anomalies. Nonetheless, as indicated before, residual analysis and correction is an arduous but necessary optimization task.
7. Independent Validation: Once the draft model is ready, it's good to have a comparables-based sample (where the sales ratios are either below 80 or above 120) worked up by an experienced appraiser or self-directed comps-based valuation site. In working up the sample, the adjustment matrix needs to be set up in line with model coefficients; for example, if a Coastal town or county is being modeled, the living area size adjustment factors would be significantly higher than their Midwest counterparts, etc. Similarly, the time and valuation dates must adequately follow the model as well. If the model shows a 12% annual appreciation in the area and the values are being forecasted for a future date, the sample must be set up accordingly; otherwise, the AVM and the validation sample would be apples to oranges. Ideally, the model values should be within 10-15% of the comps'.
8. Hold-out Sample Testing: Once the draft model is ready, and it needs to be tested on the hold-out sample kept aside at the beginning of the modeling process. Upon application of the model on to the hold-out and the first set of hold-out sales ratios are generated, the outliers need to be removed using the same ratio range as in the modeling sample. The hold-out application results (hold-out sales ratio stats – percentile distribution, COD/COV, PRD, etc.) must be very similar to those of the primary model. If they are at variance, the modeler must start investigating. Here is where the investigation should start: The modeler must immediately ensure that the final version from the 3rd cycle is being applied. Often, the newcomers are confused about the various cycles and use a model from the interim cycle.
9. Applying on to Population: One must remember that the whole AVM exercise is to develop a model from the sold population (on average, 5% homes sell annually) to value the mutually exclusive 95% unsold population. Of course, when the model is applied, it is used on the universe (meaning both sold and unsold properties). Here is why: Since the sold population is the subset of the universe, those model values will be regenerated along with the unsold, thereby forming a reasonable basis for a successful test application. In other words, as soon as the model is applied and values generated, a small matching extract from the modeling sample should be compared, and if the values match, subject to some rounding differences, the application would be in the right direction.
The need for AVM is growing by leaps and bounds. Assessors, Banks, Mortgage Companies, Mortgage servicers, REITs, Hedge funds, SFR Rentals, Tax Appeal houses, and Law firms, etc. are all big users of the certified AVM values.
-Sid Som, MBA, MIM
homequant@gmail.com
Subscribe to:
Posts (Atom)