Additions to Model template for model scoring and interpretation

[TimCookCountyDS]

Following up on issue ccao-data/model-res-avm#470 * .
Add a link to the updated wiki for model evaluation to the annual-model-checklist:

Updated wiki should to include:

• objective functions: How to interpret model metrics, choice of objective functions - operational metrics vs m.l. metrics, train/test and generalizability
• How to interpret balance tests, callouts for single variable balance tests
• Note on comparing shaps over time (changes in fits and market)
• practical, comparing changes in townships over time (big swings?)

*( old doc in https://cookcounty.sharepoint.com/:w:/r/sites/Data-Assessor/_layouts/15/Doc.aspx?sourcedoc=%7BBF69C4EB-6F06-4B31-B28C-0300857894A1%7D&file=start%20annual%20model%20issue.docx&action=default&mobileredirect=true )

See the model template checklist here-
https://github.com/ccao-data/model-res-avm/blob/master/.github/ISSUE_TEMPLATE/annual-model-checklist.md

[TimCookCountyDS]

@wrridgeway, @wagnerlmichael

Would you guys mind taking a look at this? The goal was to write a guide to model evaluation that we could link to in the annual model checklist -

It's primarily designed to cover sample generalizability and how interpret standard m.l. metrics in the context of our assessment model. (With appropriate outlinks to interpretations of assessment metrics).

• My thought is, once this is proofed, maybe add to the wiki and link out from the model-template issue?

[wagnerlmichael]

I just read it over, it looks pretty good to me.

My thought is, once this is proofed, maybe add to the wiki and link out from the model-template issue?

Sounds good to me.

[wrridgeway]

I'll take a look today

[TimCookCountyDS]

Update for tracking purposes- need to include Billy's feedback from the word doc in our wiki.