1. What are product sense/business case/metric problems

2. 3 categories of questions and frameworks

2-1 Diagnose a problem

• Did it happened sudden or progressively?

• Whether other products or features by the same company has the same change?

• Was the decline happening is an isolated region?
• Did this change only involve iOS, Android or Web users?

• You could examine which group was the root cause of the change.

2-2 Measure success

2-3 Launch or not

• Perfect scenario is that the increases of success metric are practical significant. And we don’t see difference in guardrail metric.(It doesn’t always happen in reality.)
• Conflicting result: Translate it to the impact to users and the business.
• Short-term vs. Long-term: In long term, the benefit will outweigh the drawbacks. (In reality, we need to do some in-depth analysis.)