Synthetic Data: Breaking the Data Wall

Synthetic data is created through algorithms and mathematical models. It acts like a “stunt double” for real data, serving to supplement or replace real data, and plays an important role in fields such as artificial intelligence.

What Is Synthetic Data?

It refers to data artificially generated through algorithmic models, simulation systems, or rule engines. As high-fidelity virtual data produced via algorithms, models, or rules, synthetic data has become a key technological pathway to solve the “data bottleneck” hindering AI development, protect data privacy and security, and reduce the cost of data acquisition.

It is different from traditional “raw data” and “processed data”. It is closer to a data-generation product that uses models, rules, and computing power as production factors. Its value does not depend entirely on the data source itself, but rather on the scientific nature of the generation mechanism, the reasonableness of the scenario, and the trustworthiness of the results.

Many people, upon hearing the word “synthetic”, think synthetic data is “fake data”. But in fact, it is “accurately simulated real data”. For example, the distribution patterns of user consumption data or the peak-hour patterns of urban traffic flow – synthetic data uses machine learning models to learn these characteristics and then generates new data to ensure similarity with the real data. It is also very “clean”: it contains no real names or ID numbers, eliminating privacy leakage risks at the source.

---

The Necessity of Using Synthetic Data

The training and development of various large models (LLMs) have an ever-increasing demand for data, especially high-quality data. However, the amount of data required for training large models in the real world is becoming increasingly scarce, constraining AI development.

–Not Enough: The current demand for data from large model training far exceeds the growth rate of data. In a paper, the well-known research institution EpochAI pointed out that by 2026, large models will exhaust all high-quality data; low-quality data will be exhausted between 2030 and 2050; and all image training data will be exhausted between 2030 and 2060.

“The biggest bottleneck in AI is the lack of privacy-compliant real-world data,” said Harris, CEO of UK-based synthetic data startup Mindtech Global. “Even a simple image recognition application requires up to 100,000 training images, and each image needs to be privacy-compliant and perfectly annotated by humans. Sourcing, annotating, and cleaning real-world data is a monumental task that can take up to 80% of a data scientist’s time.”

–Not Good Enough: Real-world data (e.g., internet data) suffers from uneven quality, including errors, a lack of physical interaction information, inconsistent data formats, etc. For example, pixelated images can bias model analysis results, making it impossible to effectively use such data as pre-training data for embodied foundation models.

–Cannot Use: As data usage regulations tighten and data privacy protection laws become more robust, the protection of data involving personal privacy rights, portrait rights, real personal information, etc., has increased. Enterprises are required not to arbitrarily use such data in the development and application of large models. In addition, copyright compliance review is a hidden cost – once commercial use is involved, copyright authorization must be obtained.

Synthetic data, by “indirectly expressing real patterns”, provides a risk-controllable and more efficient data supply option for the development of large models and AI.

---

Advantages of Synthetic Data

–Data quantity: Synthetic datasets are inherently clean and consistent, significantly reducing the time spent on data preprocessing and validation. It can be generated on demand, almost ‘infinitely ‘, greatly lowering the barrier for enterprises to obtain high-quality training data, especially for computer vision projects that require massive amounts of annotated data.

–Cost: Generating synthetic data does not require surveys, interviews, or expensive sensors, greatly reducing acquisition costs. For example, obtaining a single annotated real image through data labeling services may cost $6, whereas generating an equivalently valuable annotated image synthetically costs only about $0.06, effectively lowering the threshold for AI R&D. Moreover, once the generation environment is set up, algorithm iteration can further produce data variants with very low marginal cost.

–Privacy protection: Synthetic data does not contain any information that can identify real individuals or sensitive trade secrets. It is inherently low in sensitivity, avoiding privacy leakage and compliance risks at the source. This helps expand the application space of data under the constraints of personal information protection, data security, and industry regulations.

–Controllability: Data synthesis allows precise control over data distribution, proportions, and anomalies, enabling the targeted generation of highly specific and clearly bounded training samples. Generation technologies allow researchers to precisely control data distribution, features, and outliers, thereby reducing biases present in real data and improving model robustness. By artificially increasing the amount of data for specified scenarios, these artificial data ensure the model sees a more balanced and diverse set of examples.

---

Challenges of Synthetic Data

–Data Fidelity: Synthetic data cannot fully simulate the complexity and diversity of the real world, which can affect model training and inference capabilities. An article by Duke University assistant professor Emily Wenger published in Nature pointed out that images generated by large models trained on artificial data distort pictures of dogs. The reason is that training models on synthetic data ignores outliers and deviations, causing the long tail of the original data distribution to disappear, and the model increasingly drifts away from the original data distribution.

–Data Bias: There is an inherent risk of “garbage in, garbage out”. If the base data is of low quality, the generated synthetic data will inherit and amplify the original defects. Long-term and exclusive reliance on synthetic data can lead to continuous degradation of model performance and data diversity. More seriously, if incorrect or misleading information is deliberately injected, synthetic data can inherit and even amplify such biases.

–Unexplainability: The algorithmically generated nature of synthetic data may lead models to produce more insidious errors, increasing the difficulty of tracing issues. Because the generation process is often a “black box”, it is difficult to explain the specific principles and processes of data generation, potentially raising questions about the source and quality of synthetic data.

Legal And Regulatory Issues: The regulatory framework for synthetic data is still imperfect. How to ensure compliant use of generated data and how to address new security issues arising from it are questions that require further consideration and research.

---

Methods to Address The Challenges of Synthetic Data

Technical level

We must establish strict quality control mechanisms to ensure the quality and effectiveness of synthetic data.

Evaluate the consistency and generalizability of synthetic data with real data in terms of statistical properties and data structure.

Statistical similarity / distribution similarity (e.g., histogram overlap, KS test, Wasserstein distance): compare differences in key statistics and distributions between synthetic and real data.

Correlation preservation: assess whether linear or nonlinear relationships between variables are preserved in the synthetic data.

Sample-level metrics (e.g., α-Precision / β-Recall / Authenticity): evaluate fidelity and originality from the perspective of generated sample diversity and generalization.

Evaluate whether the utility of synthetic data in typical analytical or modeling tasks can substitute for real data.

TSTR vs. TRTR comparison: compare model performance when trained on two types of data and tested on real data to measure practicality.

FI Score: assess whether synthetic data is consistent with real data in terms of feature importance ranking or weighting.

Q-Score: run multiple aggregate queries on synthetic and real data to check consistency of results.

Measure the extent to which synthetic data may expose or replicate real samples, thereby assessing privacy leakage risk.

DCR: measure the minimum distance between synthetic and real samples – smaller distances may indicate copying risk.

IMS: check whether any real records are completely copied into the synthetic data (ideal value is 0).

MIA risk: evaluate the likelihood that an attacker can determine whether a specific real data point was used for training – an empirical benchmark for privacy risk.

---

Policy Level

Governments need to introduce relevant regulations that both promote and regulate synthetic data, enabling its sound and standardized development. AI Robots Eidos notes that some countries have already begun doing so.

2022: China issued the Provisions on the Management of Deep Synthesis of Internet Information Services, systematically regulating the use of deep synthesis technology and promoting the standardized development of synthetic data.

2024: Singapore’s Personal Data Protection Commission (PDPC) released the Synthetic Data Generation Guide, providing detailed explanations of generation technologies, typical applications, generation steps, etc., offering guidance on artificial data generation​ techniques and methods, and emphasizing the importance of privacy protection and data quality control.

2024: The EU’s General Data Protection Regulation (GDPR) imposed regulatory requirements on the generation and use of artificial data. Meanwhile, the European Data Protection Supervisor (EDPS) issued guidelines on data compliance for generative AI, providing a reference for compliant use of synthetic data.

2025: The U.S. Department of Commerce released Generative Artificial Intelligence and Open Data: Guidelines and Best Practices, proposing to reduce the cost of AI innovation, including the use of synthetic data, by improving the documentation, format, and quality of government data. According to the Department of Commerce, this guide is a “Dynamic Resource” that will be continuously revised and updated based on feedback and AI technology evolution.

---

How To Create Synthetic Data? – Synthetic Data Generation Methods

Overall Overview (Methods Emerging After 2013)

Using Simulators (simulators developed after 2013): Simulate real-world scenarios with simulators to obtain simulated data. For example, in autonomous driving, simulators can be used to mimic various road conditions and driving scenes to generate simulated data.

Using Generative Algorithms: Use generative algorithms (such as GANs, VAEs, convolutional neural networks, etc.) to produce synthetic data. This approach relies on the capabilities and intelligence of algorithms and models, enabling large-scale generation of high-quality synthetic data.

–Generative Adversarial Network (GAN): a deep learning framework primarily used for generating synthetic data. It consists of a generator and a discriminator that continuously improve the generator’s ability through adversarial learning, ultimately producing high-quality synthetic data.

–Variational Autoencoder (VAE): VAE is also a deep learning model that compresses input data into a latent representation in a lower-dimensional space, and then generates new data similar to the original data from this latent representation, thereby achieving synthetic data generation.

–Convolutional Neural Network (CNN): A CNN is a deep learning model widely used in areas such as image recognition, and it can also be used for synthetic data generation. By designing an appropriate convolutional network architecture and optimizing training for specific application scenarios, high-quality synthetic data can be generated.

---

Methods for the Large Language Model (LLM) Era

—Technical Frameworks

Synthetic data generation technologies cover a variety of methods, from bootstrapped reasoning to multi-agent systems. The following are representative technical frameworks from recent years, utilized for data synthesis in various scenarios.

STaR: Bootstrapping Reasoning With Reasoning (NeurIPS 2022): generates logical reasoning data through self-bootstrapped reasoning, improving model reasoning ability.

Self-Instruct (ACL 2023): generates instruction data through model self-generation, enabling efficient alignment.

CAMEL (NeurIPS 2023): generates complex dialogue scenario data based on multi-agent interaction.

Self-Play Fine-Tuning (ICML 2024): generates high-quality data through self-play fine-tuning.

TarGEN (COLM 2024): generates task-customized data.

Scaling Synthetic Data Creation with 1,000,000,000 Personas (Arxiv 2024): generates large-scale personalized data by simulating 1 billion user personas.

Source2Synth (Arxiv 2024): generates synthetic data based on real data sources, ensuring authenticity and diversity.

SynLogic (Arxiv 2024): generates data for 35 logical reasoning tasks, ensuring data quality through rule generation and verifiers.

RAGSynth (Arxiv 2024): generates diverse datasets for Retrieval-Augmented Generation (RAG) tasks, improving retriever robustness and generator faithfulness.

High-quality/complex Instruction Generation

WizardLM (Arxiv 2023): generates high-complexity instruction data through Evol-Instruct, improving model instruction-following ability.

CodecLM (NAACL 2024): aligns models through customized synthetic data, optimizing performance on complex tasks.

Magpie (Arxiv 2024): generates alignment data through unsupervised prompting, reducing the need for human intervention.

High-quality instruction generation needs to balance complexity and verifiability. WizardLM’s iterative optimization method excels in complex reasoning tasks, while Magpie’s unsupervised generation is suitable for rapid iteration scenarios.

---

Synthetic Data Examples (For Reference Only)

For readers who prefer a hands-on example, here is a simple Python script to generate math QA pairs.

import random
import json

def generate_math_qa(num_samples, difficulty="medium"):
    """Generate math QA pairs with chain-of-thought reasoning"""
    qa_pairs = []
    operators = ["+", "-", "*", "/"]
    
    for _ in range(num_samples):
        if difficulty == "easy":
            num1, num2 = random.randint(1, 10), random.randint(1, 10)
        elif difficulty == "medium":
            num1, num2 = random.randint(10, 100), random.randint(10, 100)
        else:
            num1, num2 = random.randint(100, 1000), random.randint(100, 1000)
        
        op = random.choice(operators)
        question = f"Calculate: {num1} {op} {num2} = ?"
        
        if op == "+":
            answer = num1 + num2
            cot = f"Calculate {num1} + {num2}: Add the two numbers, {num1} + {num2} = {answer}."
        elif op == "-":
            answer = num1 - num2
            cot = f"Calculate {num1} - {num2}: Subtract {num2} from {num1}, {num1} - {num2} = {answer}."
        elif op == "*":
            answer = num1 * num2
            cot = f"Calculate {num1} * {num2}: Multiply the two numbers, {num1} * {num2} = {answer}."
        else:
            answer = num1 / num2 if num2 != 0 else "undefined"
            cot = f"Calculate {num1} / {num2}: Divide {num1} by {num2}, {num1} / {num2} = {answer}."
        
        qa_pairs.append({
            "question": question,
            "answer": str(answer),
            "chain_of_thought": cot
        })
    
    return qa_pairs

def save_qa_pairs(qa_pairs, filename="math_qa.json"):
    with open(filename, "w", encoding="utf-8") as f:
        json.dump(qa_pairs, f, indent=2, ensure_ascii=False)

if __name__ == "__main__":
    qa_data = generate_math_qa(10, difficulty="medium")
    save_qa_pairs(qa_data)

Usage instructions:

Generates 10 medium-difficulty math QA pairs with CoT reasoning paths, output in JSON format. You can adjust the num_samples and difficulty parameters to control data size and difficulty.

---

Synthetic Data Companies

Leading synthetic data companies such as NVIDIA, Scale AI, and K2view have built ecosystem advantages through their technologies, occupying leading positions in areas like large model training. For example, NVIDIA provides simulation infrastructure through its Omniverse platform; Scale AI covers multiple fields with its data labeling and synthetic data services; K2view addresses enterprise-level data integration issues with its entity-centric technology.

Other synthetic data companies enhance their competitiveness through technological innovation, scenario adaptation, and fast iteration, engaging in differentiated competition in niche areas.

Recommended Related Reading from AI Robots Eidos

Synthetic data companies provide a data foundation for the advancement and widespread application of AI technology through technological innovation, cost reduction, data filling, standard setting, and ecosystem building.

If readers have some understanding of the major synthetic data companies, they should read this article about synthetic data companies.

---

Applications of Synthetic Data

–Large model field (synthetic data for AI training): It can compensate for insufficient data samples caused by objective conditions or institutional constraints (e.g., privacy compliance). It can correct biases artificially introduced during data collection and processing at the input stage, improving the reasonableness and objectivity of data distribution. For example, during large model pre-training, artificial data is mixed with real data in a specified proportion to optimize model performance and enhance generalization. Synthetic data can also address some current model alignment issues (e.g., inconsistency in human response standards). High-quality synthetic data obtained through knowledge distillation from high-performance models can improve the alignment capabilities of various models and promote model safety.

–Humanoid robot training: Synthetic data addresses the shortcomings of real data collection by generating large-scale, diversified training data at low cost and high efficiency, accelerating the transition of humanoid robots from laboratory prototypes to commercial applications. For example, large-scale synthetic data can be generated through high-fidelity physical simulation engines to train an end-to-end visual-language-action (VLA) large model for humanoid robots. This enables zero-shot generalization without real-world data, allowing the robots to perform tasks such as grasping in unseen environments.

–Financial field (synthetic financial data): By constructing synthetic financial market data and adversarial environments, synthetic data helps investment institutions predict asset price trends. Its solutions can simulate extreme market conditions and test the resilience of trading strategies to rare “black swan” events. Generated data is used to generate virtual transaction records to train anti-fraud models, significantly improving accuracy without accessing real customer data.

Synthetic data provider Hazy provided Accenture with a generative model for bank transaction data. The data generated by this model does not contain any existing customer information, but it produces realistic customer profiles and behaviors. Using synthetic data, Accenture quickly launched the project, built a prototype application for testing, and significantly accelerated the delivery of Fostering Better Finance.

–Agriculture: African agriculture has long suffered from data scarcity. The SAGDA open-source toolkit generates synthetic agricultural data for African farms, addressing the lack of soil, crop, and climate data. In a Kenyan maize yield prediction project, synthetic data augmentation improved model accuracy by 35%. The UN Food and Agriculture Organization has listed it as a “key digital agriculture infrastructure.”

–Urban governance: Singapore uses synthetic population movement data to optimize urban transportation planning, avoiding the collection of residents’ real location information. The system generates movement trajectories for 3 million virtual citizens to simulate the impact of subway line adjustments on commuting patterns, increasing planning decision efficiency by 60%. （The data is derived from Singapore’s digital twin project, “Virtual Singapore.”）

Market Research

2025 Global Synthetic Data Market Structure

Data Type	Market Share	Main Application Areas	Technology Representative
Tabular Data	42%	Finance	LLMSYNTHOR
Image/Video Data	35%	Manufacturing	3-Dimensional Modeling Engine
Text Data	18%	Large Model Training	SynthLLM
Others	5%	Intelligent Customer Service	Generative AI

Thanks to its mature technology ecosystem, strict data regulations, and early active enterprise adoption, according to a 2024 Morgan Stanley AlphaWise Survey, synthetic data solutions have the highest penetration rates in North America (approximately 35-40%) and Europe (approximately 25-30%). The current penetration rate in the Chinese market is about 20-25%. Driven by a large internet user base, rich application scenarios, and strong policy support, the penetration rate is expected to grow rapidly.

Future Development Directions for Synthetic Data

–Improve the authenticity of synthetic data: By optimizing the quality of training data, selecting advanced generative models, and combining real data for mixed training, the authenticity of generated data can be enhanced. For example, when choosing models, select appropriate models based on the data type and application scenario; for instance, CTAB-GAN+ can be used for tabular data, and TimeGAN can be used for time series data.

–Enhance the controllability of synthetic data: In practical applications, generated data must meet specific requirements and constraints. To this end, reinforcement learning (RL) can be combined with generative models. By designing learning reward mechanisms, the generative model can be continuously guided to generate data toward predetermined goals (e.g., specific style, theme, or structure), improving the practicality and targeting of the data.

–Improve privacy protection technologies: Protecting user privacy and data security during data synthesis is critical. Adopt a technology combination of “differential privacy + federated learning”. Use differential privacy (DP) to ensure the contribution of individual data points cannot be identified, thereby protecting individual privacy. Meanwhile, federated learning (FL) allows models to be trained across multiple devices or servers without data leaving the local environment, further reducing data leakage risk. Combining these two technologies enables efficient data synthesis and model training while ensuring data privacy.

–Optimize computational efficiency of data synthesis: Data synthesis often requires substantial computational resources. Therefore, data synthesis techniques with low computational cost need to be explored. For example, use “lightweight generative models” that reduce model parameters and computation while maintaining performance; use “model compression techniques” to reduce model size and improve operational efficiency. These methods enable efficient operation of data synthesis technologies even in resource-constrained environments.

Image Credits: Adlittle & Argilla & Deltalogix & Medium & Spotintelligence & Sciencedirect