Developer Tea

The Dangers of Measuring Side Effects

Episode Summary

You want to measure what matters. But your measurements might change what matters. Understanding that anything that is measured might modify behaviors, it's important to then understand what should be targeted. How can we incentivize the right actions that produce the side effects we want, rather than incentivizing direct manipulation of those side effects?

Episode Notes

You want to measure what matters. But your measurements might change what matters.

 

Understanding that anything that is measured might modify behaviors, it's important to then understand what should be targeted. How can we incentivize the right actions that produce the side effects we want, rather than incentivizing direct manipulation of those side effects?