The Skills Dashboard provides a summary of your AI assistant's readiness and performance across different skills.
Dashboard Metrics
The dashboard displays:
AI Readiness Score
An overall percentage indicating how prepared your AI assistant is to answer customer queries effectively.Predefined Questions Reviewed
Total number of predefined questions that have been reviewed.Data Sources Count
Total number of FAQs, links, documents, and other data sources added to improve AI accuracy.Unsatisfactory Responses Count
Number of AI responses that require review or improvement.
Right Panel
The right panel provides quick access to the following sections:
All Skills Dashboard
Displays an overview of the AI Skills dashboard.
All Data Sources
Opens the Data Sources page, where you can manage FAQs, text documents, PDFs, links, discounts, support tickets, and other data sources.
Unsatisfactory Responses
Displays the list of unsatisfactory responses, grouped by customer intent.
AI Skills
Order Handling
Overview – Displays the setup tasks that need to be completed.
Knowledge – Shows data sources and training template questions related to order handling intents, such as Order Cancellation and Order Modification.
Settings – Configure order tracking and tracking enrichment settings.
Shipping & Returns
Overview – Displays the setup tasks that need to be completed.
Knowledge – Shows data sources and training template questions related to shipping and return intents, such as Shipping Policy and Returns & Refunds.
Stores & Offers
Overview – Displays the setup tasks that need to be completed.
Knowledge – Shows data sources and training template questions related to store information, sales and offers, payment methods, and contact information.
Product Guidance
Overview – Displays the setup tasks that need to be completed.
Knowledge – Shows data sources related to product guidance intents, such as Product Details.
Settings – Configure product-related features, including Manage Products, Metafields, Out-of-Stock Products, Reviews, Product Rules, Recommendation Mode, and Customizations (such as Add to Cart and Follow-up Questions).
Suggested Actions
The Suggested Actions section helps merchants complete the setup of each assistant skill. A skill section is automatically removed once all required actions for that skill are completed.
1. Setup Product Guidance Skill
This skill contains the following setup steps:
Step 1: Sync Product Metafields from Shopify
Click Go to Settings.
Select Sync Meta-fields.
Choose the metafields you want to sync.
This step is marked as completed once metafields have been successfully synced.
Step 2: Add FAQs, Links, and Custom Data Sources
Click Go to Data Sources.
You will be redirected to the Knowledge Base → Product Guidance → Data Sources section.
This step is marked as completed once you visit the Data Sources page.
Step 3: Setup Product Recommendation Rules
Complete this step by either:
Creating at least one product recommendation rule, or
Clicking Skip for Now.
If Skip for Now is selected, the step is temporarily marked as completed but will reappear in Suggested Actions after all other setup steps are completed.
2. Setup Order Handling Skill
This skill contains the following setup steps:
Step 1: Setup Order Tracking
Click Setup.
This step is marked as completed once the Order Tracking page is visited.
Step 2: Review AI-Generated Responses
Click Review Responses.
You will be redirected to Knowledge Base → Review Responses.
This step is marked as completed once the page is visited.
Step 3: Add FAQs, Links, and Custom Data Sources
Click Go to Data Sources.
You will be redirected to Knowledge Base → Data Sources.
This step is marked as completed once the page is visited.
3. Setup Shipping & Returns Skill
This skill contains the following setup steps:
Step 1: Review AI-Generated Responses
Click Review Responses.
You will be redirected to Knowledge Base → Review Responses.
This step is marked as completed once the page is visited.
Step 2: Add FAQs, Links, and Custom Data Sources
Click Go to Data Sources.
You will be redirected to Knowledge Base → Data Sources.
This step is marked as completed once the page is visited.
4. Setup Store & Offers Skill
This skill contains the following setup steps:
Step 1: Review AI-Generated Responses
Click Review Responses.
You will be redirected to Knowledge Base → Review Responses.
This step is marked as completed once the page is visited.
Step 2: Add Discount Data Sources
Click Go to Data Sources.
You will be redirected to Knowledge Base → Data Sources, where the Discount Data Source modal will automatically open.
This step is marked as completed once the page is visited through this action.
Step 3: Add FAQs, Links, and Custom Data Sources
Click Go to Data Sources.
You will be redirected to Knowledge Base → Data Sources.
This step is marked as completed once the page is visited.
Suggested Actions Completion Logic
After all skill setup steps have been completed:
Any remaining Unsatisfactory Responses will appear as action items in the Suggested Actions section.
If the Product Recommendation Rules step was previously skipped, it will reappear as a suggested action until at least one rule is configured.
Once there are:
No unsatisfactory responses requiring review, and
At least one product recommendation rule configured,
the Suggested Actions section will be automatically hidden from the dashboard.
AI Readiness Score Weightage Breakdown
The overall AI Readiness Score is calculated using the following training components and weightages.
Training Component Category | Max Contribution to Overall Score | Component | Internal Weightage within Category | Calculation Logic |
1. Product Guidance | 30% | Unsatisfactory Responses | 60% | Scoring Based on >10 Clusters |
|
| Sync Metafields | 30% | 30% if synced; 0% if not |
|
| Product Recommendation Rules | 5% | 5% if Rule is Set Up |
|
| Setup Product Quiz | 5% | 5% if Quiz is Setup |
2. Order Handling | 30% | Order Tracking Setup | 40% | 40% if the page has been visited |
|
| Unsatisfactory Responses | 35% | Scoring Based on >10 Clusters |
|
| Static Predefined Questions | 25% | (Questions Answered/Total Static Qs)×25% |
3. Store & Offers | 20% | Unsatisfactory Responses | 50% | Scoring Based on >10 Clusters |
|
| Static Predefined Questions | 50% | (Questions Answered/Total Static Qs)×50% |
4. Shipping & Returns | 10% | Static Predefined Questions | 50% | (Questions Answered/Total Static Qs)×50% |
|
| Unsatisfactory Responses | 50% | Scoring Based on >10 Clusters |
5. Tagged Documents | 10% | Data Sources Quality | 100% | (Documents Tagged/Total Documents)×100%* |
Unsatisfactory Responses Scoring Model
The Unsatisfactory Responses component uses the following stepped scoring model:
Number of Clusters with > 10 Unsatisfactory Questions | % Value of Unsatisfactory Block |
0 | 100% |
1-2 | 80% |
3-5 | 60 % |
6-8 | 40% |
9-10 | 20% |
> 10 | 0% |
Overall AI Readiness Formula
The final AI Readiness Score is calculated by combining the weighted contribution from:
Product Guidance (30%)
Order Handling (30%)
Store & Offers (20%)
Shipping & Returns (10%)
Tagged Documents / Data Sources Quality (10%)
Overall AI Readiness (%) = Product Guidance Score + Order Handling Score + Store & Offers Score + Shipping & Returns Score + Tagged Documents Score
Each category score is calculated based on its internal component weightages and completion status.
Accuracy Report
Intent-Level Breakdown of AI Accuracy
This report provides a detailed breakdown of your AI assistant's performance for each customer intent.
The table includes the following columns:
Customer Intent – Displays the list of customer intents.
Training Questions – Shows the predefined questions associated with each intent.
Correct Responses – Displays the number of predefined questions that have a reviewed correct response.
Correctness – Shows the overall accuracy for each intent based on the reviewed responses.
Improvements – Displays the number of unsatisfactory responses for the intent.
Actions – Opens the Unsatisfactory Responses view, where you can review and improve the responses for the selected intent.
FAQs
1) How often is the AI Readiness Score updated?
The AI Readiness Score updates automatically as you complete setup tasks, review AI-generated responses, configure skills, or add and improve training data.
2) What does the Accuracy Report show?
The Accuracy Report provides an intent-level breakdown of your AI assistant's performance, including reviewed questions, correctness, unsatisfactory responses, and quick actions to review conversations.
3) Why are Suggested Actions still visible after I completed the setup?
Suggested Actions remain visible until all required setup steps are completed. They may also reappear if there are unresolved unsatisfactory responses or if Product Recommendation Rules were previously skipped.
For further assistance or to raise feature requests related to Skills Management, please contact [email protected].