Case study / AI assistant

Shopify Merchant Support Agent

An assistant that answers Shopify merchants’ questions from Shopify’s own documentation, and can look at the merchant’s real store data.

Code

The problem

Shopify merchants ask the same questions again and again. The answers exist, spread across the help centre, the manual and the developer documentation.

What it had to do

Answer from Shopify’s own documentation
Be fast enough to feel like a conversation
See how the merchant’s own shop is doing

What was built

A chat assistant that answers from a knowledge base built out of Shopify’s documentation. It reads real store data through a Shopify app connection, and a dashboard shows how it is being used.

How it works, step by step

Shopify’s documentation is scraped, cut into chunks and stored as embeddings in Pinecone.
A question is classified by intent: fast rules first, the model only when the rules are unsure.
The knowledge base is searched by meaning (Pinecone) and by keyword (FlexSearch) in parallel, and the results are merged.
Gemini writes the answer from the passages that were found, using the conversation so far.
When the answer’s confidence is low, the matching tool is called before replying.
Embeddings and answers are cached, so a repeated or similar question comes back at once.

My part

Most of the AI side: scraping, chunking and embedding the knowledge base into Pinecone.
Hybrid search, conversation history, intent routing and caching.
The tools the assistant can call, built on the Model Context Protocol: a calculator, web search, date and time, currency conversion, Shopify’s status page, theme compatibility and code validation.
The merchant dashboard in Tailwind CSS.
The latency analysis, then caching and running the slow steps in parallel.

The rest of the team

A teammate connected the Shopify Admin API, so the assistant can read real store data.

Decisions that shaped it

Search by meaning and by keyword together
Product names and API terms need an exact keyword match, a loosely worded question needs a search by meaning. Both run at once and the results are merged.
Rules first, the model second
The analysis put the model call that works out what a question is about at roughly 0.8 seconds. Clear cases now go through fast rules, and the model is called only when the rules are unsure.
Measure before optimising
A step-by-step breakdown of one answer showed which steps were waiting on each other for no reason. Those are the ones I ran in parallel or cached.

The outcome

A working assistant, with its architecture, data pipeline and caching documented in the repository.

The repository’s latency analysis puts one uncached answer at about 3.5 seconds, and expects 1.2 to 1.8 seconds after the changes. Both are estimates from per-step timings, not a measured benchmark.

What you can check

Code: The project folder on the branch that holds the finished work.
Open: Code, Shopify Merchant Support Agent (opens in a new tab)
Latency analysis: The step-by-step breakdown the timing figures come from.
Open: Latency analysis, Shopify Merchant Support Agent (opens in a new tab)
Who did what: The folder’s commit history: which commits are mine and which are my teammates’.
Open: Who did what, Shopify Merchant Support Agent (opens in a new tab)