When performing an A/B test, it is often the case that we randomize on one unit, for example users, but perform analysis for a metric that is computed from observations with a different unit, for example sessions. One way to solve this is to use the delta method, and this article will give an example of that.

Some information about these pages and me.

An article on the potential outcomes framework, the lingua franca for treating causal questions that sets up the theoretical foundations of causal inference. Contains references to relevant resources for those who want to go deeper.

What to do when you have to perform an experiment, but randomizing over the units that receive the treatment actually has a confounding effect on the outcome due to network effects? Or you may need to test a treatment on the same user, for example? One common answer to this question is switchback experiment design. This post is an overview of one of the rare papers I managed to find that attempt to make this approach theoretically sound.

Having determinism and free-will in the same cup is like oil and water, and has been for centuries. This is a thought experiment I have often run in my head to try to make sense of the issue while living my daily life freely.