CXSimulator: A User Behavior Simulation using LLM Embeddings for Web-Marketing Campaign Assessment

Akira Kasuga   Ryo Yonetani  

CyberAgent, Inc.  

CIKM 2024

---

📌 Overview

CXSimulator framework uses LLMs to represent user behavior events as semantic embeddings and predicts transitions between these events. This enables simulation of user reactions to new campaigns, eliminating the need for costly online testing and providing valuable insights to marketers.

🛠 Prerequisites

Operating System	Based on
Debian GNU/Linux 12 (bookworm)	python:3.10-bookworm

Software	Install
Python >= 3.10,< 3.12	-
Poetry >= 1.8.0	installer
pre-commit >= 3.8.0	pip install pre-commit

Cloud Infrastructure	Link	Summary
Cloud BigQuery	Google Analytics Sample	The dataset provides 12 months (August 2016 to August 2017) of obfuscated Google Analytics 360 data from the Google Merchandise Store , a real ecommerce store that sells Google-branded merchandise,
AzureOpenAI	Generate embeddings with Azure OpenAI	An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text.

🔧 Setup

poetry install

🚀 Getting started (Using Cache Data)

Help

poetry run python -m cxsim --help
poetry run task --list

Preprocess and Train

poetry run task model_using_cache

Simulation

poetry run task simulation_using_cache

📊 Execute All Steps

Environment Setting

Important

Authentication for cloud services is a prerequisite for executing all steps and may incur some costs.

Google Cloud

1. Enable BigQuery API in your project.

2. Install gcloud CLI

3. Auth Login.

gcloud auth application-default login

Microsoft AzureOpenAI

1. Copy template

   cp ./src/cxsim/config/.env.template ./src/cxsim/config/.env

2. Add the following content to the .env file:

    # Azure OpenAI
    AZURE_OPENAI_US_ENDPOINT=XXXXXXXX
    AZURE_OPENAI_US_VERSION=2024-03-01-preview
    AZURE_OPENAI_US_KEY=XXXXXXXX
    # Google Cloud
    GOOGLE_CLOUD_PROJECT_ID=XXXXXXXX

Preprocess and Train

Note

Once you've completed poetry run task model_using_cache, you can skip this step. In the next step, you'll simulate your campaigns using pre-trained models.

poetry run task model

Simulation

poetry run task simulation --campaign-title "Enjoy 1 month Free of YouTube Premium for Youtube related Product"

If you would like to new data period,

poetry run task simulation_for_new --campaign-title "Enjoy 1 month Free of YouTube Premium for Youtube related Product"

📄 Citation

@inproceedings{kasuga2024CXSimulator
  title={CXSimulator: A User Behavior Simulation using LLM Embeddings for Web-Marketing Campaign Assessment},
  author={Akira Kasuga and Ryo Yonetani},
  booktitle={Proceedings of the 33rd ACM International Conference on Information and
Knowledge Management (CIKM ’24)},
  year={2024},
  url={https://github.com/CyberAgentAILab/CXSimulator.git},
  doi={https://doi.org/10.1145/3627673.3679894}
}

License

This project is licensed under the Apache License 2.0.