How foodpanda tested the order of its menus
In 2023 foodpanda's data team explained how it A/B tested the order of categories and dishes in each restaurant's menu, and why the hard part was keeping 200,000 updates fresh. A second post showed how it judged the ranking model.
The first dishes on a menu get the most looks. foodpanda's data team treated their order as something to test, restaurant by restaurant: "To find the best order in which a user is presented choices, for each particular vendor, we run A/B experiments for a set of countries." Its 2023 post, written by Tommy Lim, names two hypotheses: the left-to-right order of the categories matters, and so does the top-to-bottom order of the dishes. "What we call menu ranking refers to the sorting order of categories and products".
The problem: a different test in every restaurant
Each restaurant gets a "B version", an alternative order, worked out in BigQuery by what the post calls "business logic". An internal API sets it, "using an ordered data structure such as a list", for one restaurant and one set, its categories or its dishes, per call. The experiment platform then tracks "metrics such as click-through rates to determine the success of the B version, relative to its baseline."
The post is about the part in between: a scheduled job, run with Apache Airflow, that pushes every B version to the API. That sounds like plumbing. It decided whether the test meant anything.
How they built it
The first version was a queue. Even for "the initial 5 countries", calls went one at a time: "Executing 200,000 calls sequentially took our production server 9 hours." The whole workflow took "around 13 hours" and tied up computing slots that other jobs needed.
Slow made it wrong. Restaurants edit their menus all day. By the time a B version reached the API, "around 4% of the time", the menu "either no longer existed, or did not have the same set of categories or products that the payload contained." Each failure is a restaurant whose test group never saw the planned order. And growth would break the schedule: past "approximately 14 countries", the daily job would "spill over" into the next day.
The fix was concurrency, with a priority. The team split the data into pages in BigQuery, ran one task per page at the same time, split the work by country, gave the job its own worker pool and more CPUs. Countries with the most restaurants went first, so that "a greater proportion of vendors were less “at risk” of changing their menus before we could declare the B version." The result: "Over 2x faster run times (3 hrs 45 mins) and a failure rate of 2.2% or lower."
What it delivered, and what it did not say
The numbers are pipeline numbers. The post reports no A/B result: no change in clicks, orders or basket size, no sample size, no test length. Its "2x" has an unclear baseline: 9 hours to 3 hours 45 minutes is 2.4 times faster, and the 13-hour workflow to 3 hours 45 minutes would be 3.5 times. Failures fell from about 4% to 2.2% or less, roughly 45% fewer broken tests.
A May 2023 follow-up by Grace Chia fills in the goal. The team watches "Menu Conversion Rate, i.e. the proportion of customers who had successfully ordered at least one item out of all customers who had viewed the menu", and "Average Food Value (AFV), which is the average basket value of an order, excluding delivery fee." Before a live test, it replays past orders, "with the assumption that if a user orders a particular product, then that product is more relevant than the products that were not ordered", and scores each ranking with MAP@K and DCG@K, two measures of how high the ordered dishes sit. It names its own weak spot: "there could have been a bias in product positions that affects the conversion rate". Dishes already at the top get ordered more because they are at the top. Hence: "backtesting doesn't replace the need for experiments."
What has changed since
Delivery Hero, foodpanda's parent, kept building ranking. It reported that a LambdaMART model for ranking restaurants for new users "was A/B tested and demonstrated superior performance compared to the baseline model" (2023). In March 2026 it rewrote its ranking service from Java to Python, dish ranking included, and reports five times less development time for rolling dish ranking out to a new country. In April 2026 it described a single food catalog whose "Menu Intelligence" offers "Structural suggestions for better menu organization, improving item conversion rates". And in August 2026 it launched an AI assistant that "can flag dishes and products that aren't selling well". The 15% rise in orders in that release is for Glovo, with no method given.
The company changed too. foodpanda left Thailand in May 2025. In March 2026 Grab agreed to buy foodpanda Taiwan, and Taiwan's regulator has extended its review to 27 October 2026. In July 2026 Uber announced a takeover offer for Delivery Hero at EUR 41.50 a share, foodpanda's Asian markets included, with closing expected in the second half of 2027.
Why it matters for your business
foodpanda itself left Indonesia in 2016. Its last notice to customers read: "Senin, 3 Oktober 2016 (hingga jam 10 malam) akan menjadi hari operasional terakhir kami" (Liputan6). The platforms that matter here are GoFood, GrabFood and ShopeeFood. Gojek's engineers have published one real test of ranking, for restaurant search rather than menus: "We ran an AB test using this formulation of learning to rank and observed a relative lift of 20% in search to ordering conversions and 23% improvement in NDCG" (2019).
None of the three says how it orders dishes inside a menu. What a merchant controls is categories, names and photos. GoFood marks up to five dishes in a menu's top 10% of sellers as "paling laku" (GoFood Merchant) and advises "usahakan untuk tidak membuat foto makanan dengan AI" (GoFood Merchant). ShopeeFood caps dish names at 100 characters (policy).
On your own app or website the whole order is yours to test, and the same goes for a retailer's catalog or a dealer's list of models. Ordering by outlet needs no personal data. Ordering per customer uses theirs, which Indonesia's personal data protection law (UU No. 27 of 2022) covers, so record what you use and why.
How we would build it today
- Treat the order as a ranking. Score each dish by its chance of being ordered when the menu is viewed. Start with recent best sellers per outlet and time of day.
- Judge offline, decide live. MAP@K and DCG@K on past orders are a health check. The verdict is an A/B test on menu conversion and average basket in rupiah, with guardrails on margin, cancellations and sold-out items.
- Measure position bias. Log the position of every dish shown, and shuffle a small slice of traffic, so you learn what customers want, not what you already put on top.
- Rules before models for small brands. Terlaris, Paket Hemat and Promo up front; a breakfast order and a dinner order; Ramadan and payday weeks treated as their own seasons.
- Update on change, not in a nightly batch. Push a new order when a menu changes, keyed by the menu's version, and retry when the version is stale. foodpanda's failures came from menus edited mid-run.
- Write dishes the way people search. Indonesian names such as ayam geprek or nasi uduk, clear descriptions, real photos; tag halal, spice level and bundles with a language model and check them by hand.
The lasting lesson from foodpanda: a test is only as good as the version customers actually see. Count how many of your B versions landed before you read the result.