👋 Hey, Jon here! The most viewed newsletter in the history of The Data-Driven Trades was our first analysis of AI CSR performance, and this week we’re diving back into the topic, covering ZyraTalk’s AI CSR performance from January 26th - February 26th with benchmarks and comparisons to human CSR performance. This week’s sample has nearly 3x the lead volume measured compared to the first newsletter on this topic!
Note: We had some really great questions come up the first time we published this data, so I wanted to call out a few things to keep in mind:
(1) The booking rates listed are a raw booking rate - the calculation is done using all unique leads (60 seconds or more for calls), we aren’t throwing leads away in this case even if they’re marked as “non lead”. The idea here is, if you get 100 calls answered by an AI or human CSR, how many are you booking on average?
**We are hard at work building out lead grading powered by AI with our propietary technology so we’ll have much more information about lead quality intergrated into our platform for revenue flow analysis, but for now, we’re sticking with the raw booking rate.
If you’re curious, here’s a small preview of what we’re building out:
Get in touch with us here if you want to get on our waitlist and be the first to access this product once it’s live!
(2) We aren’t currently differentiating after-hours calls vs. business hour calls - it’s reasonable to speculate on the lead quality differences, but that is not controlled for.
Recap: Below is a recap of December 1st through January 21st AI CSR performance:
Unique Lead Volume: 1,800 unique leads handled by AI CSRs
Raw Booking Rate: 36% (percentage of all unique leads that turned into a booked customer)
Run Rate: 29% (percentage of all unique leads that turned into a run job)
Paying Customer Rate: 17% (percentage of all unique leads that turned into a paying customer)
Closed Revenue: $752,000 (19% of total closed revenue handling 30% of unique lead volume)
The below data for Jan 26th through Feb 26th is from 23 different businesses across the US utilizing Zyratalk AI CSRs.
AI CSRs Drove a 36% Raw Book Rate Across 5,003 Unique Leads
Out of ~5,000 unique leads, ZyraTalk’s AI CSRs converted 36% of those to booked appointments, compared to 36% from human CSRs.
In this same time frame, human CSRs handled nearly 6,000 leads.
Last month, human CSRs handled 70% of the lead volume vs. AI CSRs handling 30%, but this month, it’s a 54% (human CSR) to 46% (AI CSR) split.
Keep in mind, the 36% a “raw booking rate” - on both the human and Zyratalk’s AI CSR side, no calls are thrown out. There are no “not leads” or “excused/abandoned calls.”
33% of the booked appointments by Zyratalk’s AI CSRs were for HVAC service, 28% were for plumbing service, and 7% were for HVAC installations.
The remaining 32% was spread out across a few categories, including a small % in water heater installs, other plumbing installs, sprinkler, electrical, and excavation.
It’s clear the Zyratalk’s AI CSRs could handle booking appointments from a diverse array of services.
28% of the booked appointments from human CSRs were for HVAC service, 22% were for plumbing service, 8% were for HVAC sales/install, and 7% were for electrical service. The remaining ~35% was spread out over a few different services.
Of the leads booked by ZyraTalk’s AI CSRs, 8% of them were new customers compared to 16% for human CSRs.
Unique Leads Handled By AI CSRs Had A 28% Run Rate
About 5% of the unique leads booked by ZyraTalk’s AI CSRs canceled their appointments, compared to about 5% from human CSRs.
So far with a larger call volume evaluated, human and AI CSRs both had the same raw booking rate and cancel rates, although human CSRs handled about 1,000 more leads than the AI CSRs.
The initial key metric we care about with CSRs is the book rate, but we do want to see how those leads fare as they get further down the funnel.
For example, if there was a major drop off (cancellations) between booked job rate and match/run rate, there might be an issue in how those appointments are booked, but both are seeing a normal level of appointment cancellations so far.
22% of jobs run from ZyraTalk’s AI CSR bookings were from new customers, compared to 39% from human CSRs.
Unique Leads Handled By AI CSRs Had A 16% Paying Customer Rate
16% of of the total unique leads handled by ZyraTalk’s CSRs turned into paying customers with an average ticket of $2,964.05.
22% of those paying customers were new and spent an average of $3,732.77.
The paying customer rate of unique leads booked by human CSRs was 3% higher, coming in at 19%, with an average ticket of $3,759.05.
Of those paying customers, 37% were new and spent an average of $4,113.78.
Paying customer rates are more reliant on the in-home sales process than they are CSRs, but when doing any analysis, it is important to follow the customer journey all the way through to revenue.
Let’s take a look at the revenue generation comparison between ZyraTalk CSR’s and human CSR’s 👇
Unique Leads Handled By AI CSRs Generated $1.86 Million of Closed Revenue
Out of the total lead volume in this data sample, ZyraTalk’s AI CSRs handled 46% of that volume, generating $1.86 million in closed revenue, or 35% of the total closed revenue.
Unique leads booked by ZyraTalk’s AI CSRs also generated $4 million in revenue opportunity (estimated revenue + sold revenue + closed revenue), which was 33% of the total revenue opportunity generated.
62% of the AI CSR revenue opportunity converted to sold/closed revenue vs. 58% of the human generated revenue opportunity.
In comparison, the human CSRs, handling 54% of the lead volume, generated 65% of closed revenue for the businesses in this sample.
Although book and cancel rates were the same across CSRs, the biggest difference were the average tickets, again favoring human CSRs with nearly $800 in additional spend per paying customer, and 8% more lead volume.
Of the closed revenue generated by ZyraTalk’s AI CSRs, 25% came from new customers, compared to 39% from human CSRs.
Until next time . . .
-Jon