Cohort Analysis using AI

The feedback on my last article about using ChatGPT for Churn Analysis was quite positive, so I thought I'd keep playing around and see what else can be done.

In fact, AI has also proven useful in performing a SaaS cohort analysis.

Refresher on Cohort Analysis

Data used + Restrictions

For my analysis, I used the sample data from Point Nine's guide. You can usually extract transaction data like this from your billing platform (Stripe, Chargebee, Paddle,..).

If you are unfamiliar with pivot tables and have perhaps never performed a cohort analysis before, AI now comes to the rescue.

Please note, however, that the results are based on a stochastic model and should be checked for plausibility in any case.

Think of it more as a “first cut” than as auditable figures.

Right-aligned Cohort Analysis

I started by importing the sample data set and creating a table that shows me:

1. Individual cohorts (grouped by sign-up month) in the first column,

2. Total transaction amount of each cohort per calendar month from the earliest to the latest transaction date in the columns right next to it,

3. Sum of the transaction amounts of all cohorts in each month in the bottom row.

After some trial and error, I found a prompt that generated what I wanted quite reliably.

The resulting table is a right-aligned cohort analysis that shows the effects of stacking revenue cohorts over time.

I find that presenting the data of a right-aligned cohort analysis in a chart instead of just a table gives a much clearer picture.

Therefore, I created a stacked area chart, also called a layer cake chart, to present the table data in a more understandable way.

I must say that it took me a while to get there and despite trying many different prompts, I often faced errors and found it hard to get consistent results.

This was a bit frustrating and eventually led me to try another tool that gave better and more reliable results (more on this below).

Left-aligned Cohort Analysis

The left-aligned variant is actually more common. It shows how individual cohorts develop over the course of lifetime months rather than calendar months. This makes it easier to compare individual cohorts with each other.

You can also adjust the table to display percentages, with each cohort's initial month transactions as 100% and subsequent columns showing transaction volumes as a percentage of the first month's total.

This will allow you to better identify whether your cohorts are growing or shrinking over time, i.e. how much of each cohort's originally acquired revenue you can retain.

If you want, you can of course also let the AI interpret the results. I'll leave that up to you. Since the sample data does not come from a real SaaS business, it makes little sense here.

Alternative to ChatGPT

As you can see above, I ended up getting the results I wanted, but spent quite a lot of time fiddling with prompts.

Prompt engineering significantly improved reproducibility, but ChatGPT still did not always deliver what you would expect.

That's why I switched to a tool that was recommended to me and that is better suited for these types of analyses: Julius AI.

It seems like the tool does a better job of capturing what it's supposed to do / what result I expect - often with less context than I gave ChatGPT.

As you can see below, Julius AI also generates much better diagrams off the cuff.

Feel free to try it out for yourself, it’s free.

Key findings

1. AI can give you a jump-start with your cohort analysis - especially if you are not familiar with spreadsheet programs and/or have never performed such an analysis before.
   

2. Depending on the prompts, the reproducibility is okay, but not outstanding. Results should always be checked for plausibility.
   

3. It may make sense to look beyond ChatGPT for AI tools like Julius AI that are better geared towards data analysis.

PS: Hosting an AMA session next week with Ben Murray aka “The SaaS CFO” on SaaS Finance & Metrics. Would love to see some of you there!

You can register here.