04-27, 11:45–12:30 (America/Los_Angeles), St. Helens
We will discuss industry best practices for leveraging experimentation by product development teams. We'll cover how to make advanced statistics accessible so that cross-functional stakeholders can translate results into action. We'll also share the secrets for scaling experimentation to thousands of simultaneous experiments, an achievable goal for teams of any size.
Experimentation is the scientific "gold standard" of measuring causality and has proven to be the most powerful data tool for product development teams at companies such as Meta, Amazon, Netflix, and Snapchat. This talk outlines how to design a stats engine and scale an experimentation platform. The following topics will be covered:
- Selecting statistical techniques and tools which balance pragmatism and statistical rigor.
- Discussing Frequentist vs Bayesian, why a 95% confidence interval works so well, and how to carefully select experimentation metrics.
- Making advanced statistics accessible to cross-functional stakeholders, typically engineers and PMs.
- Driving the flywheel of ideas and experimentation, and how to scale experimentation to hundreds of simultaneous experiments.
Previous knowledge expected
Timothy Chan is an experienced data science professional, currently serving as the Data Science Lead at Statsig. This cutting-edge platform provides product observability and experimentation services to top companies such as Notion, RecRoom, Univision, and Ancestry. Before joining Statsig, Timothy spent almost 5 years as a Data Scientist at Facebook (now Meta), where he was involved in projects across Facebook App and Reality Labs. Before venturing into the tech industry, Timothy worked in biotech, researching treatments for diseases such as Alzheimer’s, Multiple Sclerosis, Lupus, and Cancer. He holds a PhD in Chemistry and an MBA in Entrepreneurship.