SynthTRIPS: A Knowledge-Grounded Framework for Benchmark Query Generation in Personalized Tourism Recommenders

Abstract

Tourism Recommender Systems (TRS) are crucial in personalizing travel experiences by tailoring recommendations to users’ preferences, constraints, and contextual factors. However, publicly available travel datasets often lack sufficient breadth and depth, limiting their ability to support advanced personalization strategies—-particularly for sustainable travel and off-peak tourism. In this work, we explore using Large Language Models (LLMs) to generate synthetic travel queries that emulate diverse user personas and incorporate structured filters such as budget constraints and sustainability preferences.

This paper introduces a novel framework, SynthTRIPS, for generating synthetic travel queries using LLMs grounded in a curated knowledge base (KB). Our approach combines persona-based preferences (e.g., budget, travel style) with explicit sustainability filters (e.g., walkability, air quality) to produce realistic and diverse queries. We mitigate hallucination and ensure factual correctness by grounding the LLM responses in the KB. We formalize the query generation process and introduce evaluation metrics for assessing realism and alignment. Both human expert evaluations and automatic LLM-based assessments demonstrate the effectiveness of our synthetic dataset in capturing complex personalization aspects underrepresented in existing datasets. While our framework was developed and tested for personalized city trip recommendations, the methodology applies to other recommender system domains.

Code and dataset are made public at: https://bit .ly/synthTRIPS

SynthTRIPS

Figure 1 in the paper illustrates the proposed SynthTRIPS framework for generating synthetic travel queries using LLMs. The diagram highlights three key components:

Persona Hub: A structured repository of diverse user personas that influence query generation. This ensures personalization by incorporating user-specific preferences such as budget, travel style, and sustainability concerns.
Travel Filters: A set of constraints applied to queries, including budget, seasonality, walkability, air quality, and destination popularity. These filters guide the LLMs in generating queries that reflect real-world constraints.
Contextual Prompting with KB Grounding: The LLM is prompted with knowledge-grounded inputs from a curated Knowledge Base (KB) that contains factual information about destinations. This mitigates hallucination and ensures queries are realistic and contextually accurate.

The figure visually represents how these components interact, showing a flow from persona selection and filter application to structured prompting and final query generation. This framework enables the automated creation of diverse, sustainability-aware travel queries, which can be used to benchmark personalized tourism recommender systems.

Method Overview — [Fig.1] SynthTRIPS: Our proposed framework for generating synthetic data using LLMs for personalized, sustainable city trips..

Dataset

[Data Repository]

A comprehensive Knowledge Base (KB) covering 200 European cities across 43 countries.
- It includes detailed information on points of interest (e.g., attractions, activities, and destinations),
- Destination popularity
- Estimated monthly visitor footfall (seasonality) and
- Key sustainability metrics such as walkability and air quality index (AQI).
This Knowledge Base (.csv format) was used to generate queries using LLMs.

Synthetic travel queries (.json format) generated for 2302 configurations in 3 settings -- vanilla (v/Non-personalized), personalized-zero-shot (p0) and personalized-single-shot (p1) using:
- Llama-3.2-90b
- Gemini-1.5-pro-002

Prompts (both system prompts & user prompts along with examples used for ICL) used for both query generation and validation can be found here.

[Fig. 2a] Distribution of the cities in our KB, grouped by popularity levels.

[Fig. 2b] Radar Chart showing the different dimensions of validation and performance of queries generated by the two models. L (E) denotes LLM (Expert) validations.