Skip to content

eg4.md

Latest commit

 

History

History
79 lines (60 loc) · 4.3 KB

eg4.md

File metadata and controls

79 lines (60 loc) · 4.3 KB

 home :: syllabus :: timetable :: groups :: moodle(591, 791) :: video tbd :: © 2021

Eg4: Model stores

If you can do something fast, then you can also do bad things fast. For example, there are tools that can take any Github repo, container-ize it, then drop it up onto AWS as a web service. Crappy prototype to slick interface in a matter of seconds.

Machine learning methods can be packaged and executed in cloud "containers" (e.g. using tools like opendatahub.io). Many such containers are available for rent, in [model stores at AWS marketplace and elsewhere Users rent these models plus the CPU needed to run them. They upload their data and, later, the model returns labels (conclusions).

As seen in the following table, a study by Xiu et al. in Jan'21 found over 300 such models-for-rent. Given the current interest in commercial AI, it is expected that these numbers will soon grow much larger.

These models make decisions that could inappropriately affect different social groupings defined by age, gender, or race. For example, in the AWS model store,

  • Credit Default Predictor uses attributes like gender, education, age, and previous history of payments. Assessing the fairness of Credit Default Predictor is an important task since such predictors have the potential to perpetuate social inequities (e.g. it is hard to break the cycle of poverty if your zip code prevents you from access a line of credit).
  • Also in the AWS model store, the Hospital Readmission model predicts the probability that a patient will not be readmitted after discharge. This model's inputs include financial class, sex and age. It is important to assess the fairness of Hospital Readmission since it is highly inappropriate (and dangerous) to affect a patient's life expectancy due to their (e.g.) social status.

So how to trust that model stores are not dispensing discriminatory models that are unfair to certain social groups? This is an important question since:

  • Given the ubiquity of the internet, model stores can unleash discriminatory models to a wide audience.
  • Currently, the fairness of the models is unknowable. Model store models usually have no tools for (a) measuring or (b) mitigating discriminatory bias.

This second point is particularly troubling since Xiu et al. warn that over 70% of these models are early lifecycle research prototypes. As such, we should as the very least routinely apply some kind of fairness verification process to the models in model stores.

TODO

  • Read the Xiu et al. article
    • Ask what tools should be standard for models stores (w.r.t. fairness) but are not
    • What does this all say about the Thearanos example? Are we in a culture where flash matters more than substance? That if it looks good and it sells then as a model producer we are not required to more than than
  • Read a a proposal to fix unfariness
    • What does it miss? Are there better ways to do this?
    • Here's the hardest question I know. This proposal says we can make a model fairer by replacing it with a local, fairer, copy. Does this mean we cannot have an AI economy where if you build a model, you can never keep control of it and (e.g.) earn an income by selling the model services?