How DoorDash decided whether a store was really closed
In January 2023 two DoorDash data scientists described how the app handles a courier who reports a store as closed. A set of rules gave way to one LightGBM model that scores the report and picks one of three actions. The model is simple; the hard part was making labels for something nobody measures.
A courier arrives, the shutters are down, and the order is stuck. The app now has to choose: send another courier, cancel and refund, or take the store off the app so nobody else orders from it. At DoorDash this happens at volume: "thousands of closed store cases that are reported each day", against "tens of millions of deliveries being fulfilled each week". On 10 January 2023 Slava Nikitin and Gabriel Lerner described how DoorDash replaced the rules behind that choice with one small model. The model is the easy part. The lesson is how they got labels for a fact nobody records.
The problem: a report is not the truth
DoorDash built the report into the courier app as "Dasher reports store closed", or DRSC. The courier takes a photo of the store; "When a valid picture is uploaded, the Dasher is compensated for the partial time and effort they spent getting to the store", and is unassigned. DoorDash then cancels and refunds, or reassigns the order when it doubts the report.
In parallel it contacts the merchant. "If the merchant confirms the closure or is unresponsive, we will pause the store for a set period of time." If the merchant says it is open, the report is rejected and a new courier is found.
Some reports are wrong. The post names two kinds: the photo "does not show a closed store", or the merchant says it is open. A 2021 DoorDash post on its image models is blunter: "some Dashers fraudulently report a store closure by uploading a fake storefront image and then collecting half of the delivery fee."
Trust a false report and a merchant loses sales; doubt a true one and a second courier drives to a closed door. People could check every report, but "it would be expensive, time-consuming, and unscalable." Rules were the first answer: "A heuristic set of rules could handle the required speed and scale but would be only moderately accurate, with errors in passing inaccurate reports being costly to merchants and painful for customers."
How they built it
Ask one narrow question. The model computes "Probability(Store is Closed | DRSC report)": given this report, how likely is the store closed? The team stayed narrow on purpose: "While a more general model for inferring store status could be built, we constrained ourselves to only handling the DRSC use case."
Make labels from the next hour. "Our first challenge was that the dependent variable (the status of a store) is unknown and would be prohibitively expensive to measure". So the team looked at what happened after old reports: whether the store "were fulfilling orders or responding to our messages about the store status in the hour following a DRSC report." Their example: "when a Dasher is able to complete a pick up within 15 minutes of a historical DRSC report, we would infer that the store was probably still open, despite the report."
Use history, not the report alone. A report carries a store ID, a courier ID, a time and a photo. The IDs link it to history, which gives features such as "the number of recent reports for a given store, the last time a pickup occurred, and the number of invalid store-closed reports for a given Dasher over multiple months."
Shrink the photo to one number. A separate image classifier scores each photo for signs such as a closed sign or dark windows. "The image classifier compresses the image information into a single number which we then use as one of our model features." The 2021 post says those models use "ResNet as a backbone network" and can check the courier's GPS against the store's location; the 2023 post does not list GPS as a feature.
One model, three bands. "a single LightGBM model can combine historical and image information to compute a probability that a store is closed." LightGBM is an open-source library of gradient-boosted decision trees, the usual tool for tables of numbers. Fixed thresholds split the probability into actions: low means unassign and find a new courier, intermediate means cancel the order, and high means "both cancel the order and pause the store."
What it delivered
One claim, from an A/B test: "the improved decision-making now saves thousands of deliveries from being canceled every week."
"Thousands" against tens of millions of weekly deliveries is small as a share and real as a count. The post gives no accuracy, no threshold values, no test size, no feature count and no word on how fast it runs. These are DoorDash-reported claims.
One weakness is ours to point out, not the post's. The labels come from orders after a report. Once the model pauses a store, that store gets no orders, so the label it would have produced never appears. A store paused by mistake cannot prove it was open. Left alone, a system like this learns least about its own worst errors.
The post names its next step: thresholds that move with "time of day, future store volume, and potential future cancellations", set by a cost for each action. A wrong pause at the dinner peak costs more than one at 3 pm.
What has changed since
DoorDash gave merchants more ways to say they are closed. Its developer release notes added store deactivation webhooks for point-of-sale partners in February 2023 and automatic temporary deactivation for outages in June 2023. A store closed through those APIs without an end date now comes back on its own after two weeks.
The merchant check got a voice. A December 2025 intern post describes the old way: "DoorDash sends an automated phone call or robocall to the merchant, asking them to press specific numbers on a phone tree to confirm if they are open". Its limit: "The system relies solely on a simple yes/no response, which prevents us from gathering precise schedules ahead of time." The intern built a voice assistant that asks for hours in conversation. It is an intern project, and the post gives no results.
Store status is now an AI-agent problem too. DoorDash's June 2026 post on its shopping assistant lists, under grounding failures, "Stores recommended as open when they're closed". The fix: "route the agent's claim through a tool call against the system of record." An agent can only be as right as the store record it reads.
Google calls businesses as well. Its Business Profile help mentions "automated calls or texts" to confirm details, and since July 2025 Search can call businesses for pricing and availability on a user's behalf in the US.
DoorDash also grew, closing Wolt in June 2022 and Deliveroo in October 2025. We found no follow-up post on this model.
Why it matters for your business
Indonesian platforms already have the merchant-side tools. GoFood merchants can set special hours and use "Istirahat" or "Tutup Sementara". GrabFood allows up to five special schedules two months ahead, plus a temporary stop. ShopeeFood goes furthest: its outlet score counts cancels caused by the merchant, "Outlet tutup" first in the list, and the share of operating hours the outlet spent paused. That is DoorDash's cancel-and-pause logic turned into a merchant score.
The courier side is where a smaller app is weakest. A warung shut for Lebaran, a clinic that closed early: any app that sends a person somewhere needs DoorDash's three choices before a second person travels. For booking apps, warn the customer earlier still.
A door photo can show passers-by and number plates, and the data protection law, UU 27/2022, covers data that can identify a person alone or combined. Keep the photo only as long as the check needs it. That is our reading as engineers, not legal advice.
How we would build it today
- Use what you already know. Schedule, special hours, Ramadan and Lebaran, the last accepted order and whether the merchant app is online. Catch closed stores before dispatch.
- Make the report one signal. Ask for a photo, GPS and time, and check GPS against the store pin. Weigh each courier by their past report accuracy, as DoorDash does.
- Start with a rules table. Log every report and what happened in the next hour. Move to gradient-boosted trees, LightGBM or similar, once a few weeks of outcomes are logged.
- Keep three actions. New courier, cancel, cancel and pause. Keep the middle band wide at first.
- Ask the merchant with a script. A short WhatsApp or phone check with fixed answers: buka, tutup, jam khusus. Get the merchant's consent to be contacted this way.
- Protect your labels. Send some high-band reports to a call instead of an automatic pause, so a wrong pause can still show up in the data.
- Help merchants close properly. One-tap Tutup Sementara, a reminder before holidays, and a paused-time score like ShopeeFood's. Every planned closure is one report fewer.
- Price the mistakes. Put a rupiah cost on a refund, a lost sale and a wasted trip, and let those costs set the band edges. Measure closed-store cancels per 1,000 orders, wrong pauses and time to a new courier.
For an app with a few hundred merchants, steps 1, 2, 5 and 7 with a review queue are a fair start. Add the model when reports arrive faster than people can check them.