Python experiments installation

pip install posthog

In your app, import the posthog library and set your project API key and host before making any calls.

from posthog import Posthog

posthog = Posthog('<ph_project_api_key>', host='https://us.i.posthog.com')

Note: As a rule of thumb, we do not recommend having API keys in plaintext. Setting it as an environment variable is best.

You can find your project API key and instance address in the project settings page in PostHog.

Once PostHog is initialized, you should be able to capture events. For your experiment to be meaningful, we need to capture an event that we want to measure, such as a conversion event.

For this tutorial, let's report a conversion event when a clicks a CTA. In server SDKs like Node, this can be called when a specific route is visited or reported from your client side app.

posthog.capture('cta clicked', distinct_id='user_distinct_id')

Before proceeding, let's make sure events are being captured and sent to PostHog. You should see cta clicked events appear in the Activity feed.

Go to the Experiments tab in the PostHog app and click on the New experiment button in the top right.

For this tutorial, let's create a new experiment using simplified test values:

• Name: "Test experiment"
• Description: "This is a test experiment"
• Feature flag key: "test-experiment-ff-key"
• Experiment type: "Feature flag"
• Variants: "control" and "test"
• Participant type: "Users"

Then click Save as draft.

Scroll down to the Primary metrics section and click + Add primary metric.

Choose Single-use and select Type > Mean.

Then search for the event cta clicked under Metric and click Save.

By default, experiments are exposed to 100% of users. You can customize release conditions to expose the experiment to a subset of users.

For this tutorial, we'll ship the experiment to all users and click Launch in the top right.

Use the PostHog SDK to call the experiment flag and update how your page renders based on the assigned variant.

variant = posthog.get_feature_flag('your-experiment-key', 'user_distinct_id')

if variant == 'test':
    # Show test variant
else:
    # Show control variant

Now when a user triggers a cta clicked event, PostHog automatically assigns the user to a variant and records an experiment exposure.

By default, users are split equally between variants. If you want to assign specific users to a specific variant, see more about distribution and release conditions.

Make sure exposures and feature flag calls are being sent to PostHog. You should see $feature_flag_called events appear in the Activity feed.

As you capture more cta clicked events, more exposures will populate the primary metrics in your experiment.

With enough data, you can analyze the experiment and its variants by:

• Conversion rates
• Statistical significance
• Credible intervals
• Chance to win %
• Minimum detectable effect
• And more