NLP
Support Ticket Triage
- Company
DoorDash- Job positions
- Data ScientistML Engineer
- Topics
- Text classificationNLPscikit-learnModel evaluationError analysis
The scenario
DoorDash: Route customer support tickets to the right team from their text, and prove the model beats a keyword baseline.
DoorDash's support team receives thousands of customer messages a day. Every ticket lands in one queue, and an agent reads it before passing it to the right team, which slows down urgent problems like late orders. You're a data scientist on the support operations team, asked to route tickets automatically based on what customers write.
Your task
Build a model that reads a ticket's text and predicts its category, evaluate it honestly against a simple baseline, and predict the categories of the held-out tickets.
Instructions
- 1Explore the training tickets: how the categories are balanced, how long messages are, and the typos, capital letters and mixed problems real customers write.
- 2Build a keyword baseline by hand and measure it with macro F1, so every category counts equally.
- 3Turn the text into features, such as word and word-pair counts or TF-IDF, and train at least two classifiers. No large language models or paid APIs are needed.
- 4Compare the models with cross-validation, pick one, and show its confusion matrix.
- 5Read 20 of the model's mistakes and group them by cause, such as tickets that mention two problems or labels that look wrong.
- 6Recommend how the support team should use the model, including when a ticket should still go to a person.
- 7Predict the category of every ticket in tickets_test.csv and save the results as predictions.csv with the columns ticket_id and category.
Datasets
All files come in one download.
tickets_train.csv
3,000 support tickets from August and September 2025, with the category an agent assigned.
3,000 rows · 6 columns · 361 KB
| Column | Type | Description |
|---|---|---|
| ticket_id | text | Unique ID of the ticket. |
| created_at | datetime | When the customer opened the ticket. |
| channel | text | Where the ticket came from: chat, email or app_form. |
| order_value | decimal | Value of the related order, in USD. |
| text | text | What the customer wrote. |
| category | text | Team the ticket was routed to. This is what you predict. |
Preview the first 5 rowsHide preview
| ticket_id | created_at | channel | order_value | text | category |
|---|---|---|---|---|---|
| T100001 | 2025-08-07 05:44:48 | chat | 18.64 | Ugh dasher asked me to cancel the order second time this week | dasher_feedback |
| T100002 | 2025-08-03 13:37:34 | chat | 35.57 | They forogt my dumplings again. also card got declined but the money still left my account this is ridiculous | missing_item |
| T100003 | 2025-08-28 13:52:23 | chat | 41 | hey supoprt, no side of rice in my delivery thanks | missing_item |
| T100004 | 2025-09-19 11:16:39 | chat | 92.04 | UGH COMPLETELY WRONG ORDER DELIVERED TO ME SECOND TIME THIS WEEK | wrong_order |
| T100005 | 2025-09-12 03:47:12 | 96.12 | Where is my order, it said it would arrive 47 minutes ago second time this week | late_delivery |
tickets_test.csv
1,000 held-out tickets from the same period, without their category.
1,000 rows · 5 columns · 106 KB
| Column | Type | Description |
|---|---|---|
| ticket_id | text | Unique ID of the ticket. |
| created_at | datetime | When the customer opened the ticket. |
| channel | text | Where the ticket came from: chat, email or app_form. |
| order_value | decimal | Value of the related order, in USD. |
| text | text | What the customer wrote. |
Preview the first 5 rowsHide preview
| ticket_id | created_at | channel | order_value | text |
|---|---|---|---|---|
| T100009 | 2025-09-20 02:11:52 | chat | 44.92 | HI, RECEIVED GARLIC BREAD INSTEAD OF THE DUMPLINGS |
| T100013 | 2025-08-08 03:27:51 | chat | 81.07 | hey support, my order came without the sauce |
| T100018 | 2025-08-31 05:14:30 | app_form | 74.21 | oredr is 22 minutes late and still not here please help |
| T100019 | 2025-08-18 19:08:26 | chat | 102.11 | order #525473: refund was approved 4 days ago but never hit my card this is ridiculous |
| T100021 | 2025-08-28 00:20:05 | app_form | 61.3 | They forgot my side of rice again fix this asap |
Hint
Macro F1 averages the F1 score of each category, so a model that ignores small categories such as account access scores poorly even when its overall accuracy looks high.
from sklearn.metrics import f1_score f1_score(y_true, y_pred, average="macro")
Deliverable
A public GitHub repo with your notebook or scripts, predictions.csv for the test tickets, and a README with your baseline, model results, error analysis and routing recommendation.
When you're done, post your repo in the Solutions tab to share it with other learners.
What grading checks
Use this checklist to review your own work before you post and share it.
- Submitted GitHub repo is public and reachable.
- Repo contains at least one notebook or script file.
- Repo contains predictions.csv with ticket_id and category for all 1,000 test tickets.
- Predictions reach a macro F1 score of at least 0.80 on the test tickets.
- The model is compared with a keyword baseline using cross-validation.
- Error analysis groups the model's mistakes by cause.
- Recommendation explains how support should use the model and when a person should review a ticket.