# embedding_metadata

key: chroma:document
string_value:
- Yuji Wang, Jingchen Ni, Yong Liu, Chun Yuan, and Yansong Tang. Iterprime: Zero-shot referring image segmentation with iterative grad-cam refinement and primary word emphasis. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pp. 8159–8168, 2025c. 

- Cong Wei, Yujie Zhong, Haoxian Tan, Yong Liu, Zheng Zhao, Jie Hu, and Yujiu Yang. Hyperseg: Towards universal visual segmentation with large language model. _arXiv preprint arXiv:2411.17606_ , 2024. 

- Nicolai Wojke, Alex Bewley, and Dietrich Paulus. Simple online and realtime tracking with a deep association metric. In _2017 IEEE international conference on image processing (ICIP)_ , pp. 3645–3649. IEEE, 2017. 

- Junfeng Wu, Yi Jiang, Qihao Liu, Zehuan Yuan, Xiang Bai, and Song Bai. General object foundation model for images and videos at scale. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 3783–3795, 2024a.


key: chroma:document
string_value:
- Junfeng Wu, Yi Jiang, Qihao Liu, Zehuan Yuan, Xiang Bai, and Song Bai. General object foundation model for images and videos at scale. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 3783–3795, 2024b. 

- Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmentation via multimodal large language models. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 3858–3869, 2024. 

--- end of page.page_number=74 ---

- Yin Xie, Kaicheng Yang, Xiang An, Kun Wu, Yongle Zhao, Weimo Deng, Zimin Ran, Yumeng Wang, Ziyong Feng, Roy Miles, et al. Region-based cluster discrimination for visual representation learning. _arXiv preprint arXiv:2507.20025_ , 2025.


key: chroma:document
string_value:
- Hu Xu, Po-Yao Huang, Xiaoqing Ellen Tan, Ching-Feng Yeh, Jacob Kahn, Christine Jou, Gargi Ghosh, Omer Levy, Luke Zettlemoyer, Wen-tau Yih, et al. Altogether: Image captioning via re-aligning alt-text. _arXiv preprint arXiv:2410.17251_ , 2024a. 

- Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying clip data, 2024b. URL https://arxiv.org/abs/2309.16671. 

- N. Xu, L. Yang, Yuchen Fan, Dingcheng Yue, Yuchen Liang, Jianchao Yang, and Thomas S. Huang. Youtube-vos: A large-scale video object segmentation benchmark. _ArXiv_ , abs/1809.03327, 2018. 

- Yifan Xu, Mengdan Zhang, Chaoyou Fu, Peixian Chen, Xiaoshan Yang, Ke Li, and Changsheng Xu. Multi-modal queried object detection in the wild. _Advances in Neural Information Processing Systems_ , 36:4452–4469, 2023.


key: chroma:document
string_value:
- Cheng-Yen Yang, Hsiang-Wei Huang, Wenhao Chai, Zhongyu Jiang, and Jenq-Neng Hwang. Samurai: Adapting segment anything model for zero-shot visual tracking with motion-aware memory. _arXiv preprint arXiv:2411.11922_ , 2024. 

- Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. _arXiv preprint arXiv:2310.11441_ , 2023. 

- Heng Yin, Yuqiang Ren, Ke Yan, Shouhong Ding, and Yongtao Hao. Rod-mllm: Towards more reliable object detection in multimodal large language models. In _Proceedings of the Computer Vision and Pattern Recognition Conference_ , pp. 14358–14368, 2025. 

- En Yu, Tiancai Wang, Zhuoling Li, Yuang Zhang, Xiangyu Zhang, and Wenbing Tao. Motrv3: Release-fetch supervision for end-to-end multi-object tracking. _arXiv preprint arXiv:2305.14298_ , 2023a.


key: chroma:document
string_value:
- En Yu, Tiancai Wang, Zhuoling Li, Yuang Zhang, Xiangyu Zhang, and Wenbing Tao. Motrv3: Release-fetch supervision for end-to-end multi-object tracking. _arXiv preprint arXiv:2305.14298_ , 2023a. 

- Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020. 

- Seonghoon Yu, Paul Hongsuck Seo, and Jeany Son. Zero-shot referring image segmentation with global-local context features. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pp. 19456–19465, 2023b. 

- Seonghoon Yu, Paul Hongsuck Seo, and Jeany Son. Pseudo-ris: Distinctive pseudo-supervision generation for referring image segmentation. In _European Conference on Computer Vision_ , pp. 18–36. Springer, 2024.


key: chroma:document
string_value:
- Rowan Zellers, Jiasen Lu, Ximing Lu, Youngjae Yu, Yanpeng Zhao, Mohammadreza Salehi, Aditya Kusupati, Jack Hessel, Ali Farhadi, and Yejin Choi. Merlot reserve: Multimodal neural script knowledge through vision and language and sound. In _CVPR_ , 2022. 

- Fangao Zeng, Bin Dong, Yuang Zhang, Tiancai Wang, Xiangyu Zhang, and Yichen Wei. Motr: End-to-end multipleobject tracking with transformer. In _European conference on computer vision_ , pp. 659–675. Springer, 2022. 

- Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers, 2022. URL https://arxiv.org/abs/2106.04560. 

- Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. _arXiv preprint arXiv:2203.03605_ , 2022a.


key: chroma:document
string_value:
- Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq-Neng Hwang, and Jianfeng Gao. Glipv2: Unifying localization and vision-language understanding. _Advances in Neural Information Processing Systems_ , 35:36067–36080, 2022b. 

- Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. _Advances in neural information processing systems_ , 37:71737–71767, 2024a. 

- Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. In _European conference on computer vision_ , pp. 1–21. Springer, 2022c. 

--- end of page.page_number=75 ---


key: chroma:document
string_value:
--- end of page.page_number=75 ---

- Yuxuan Zhang, Tianheng Cheng, Lianghui Zhu, Rui Hu, Lei Liu, Heng Liu, Longjin Ran, Xiaoxin Chen, Wenyu Liu, and Xinggang Wang. Evf-sam: Early vision-language fusion for text-prompted segment anything model. _arXiv preprint arXiv:2406.20076_ , 2024b. 

- Zheng Zhang, Yeyao Ma, Enming Zhang, and Xiang Bai. Psalm: Pixelwise segmentation with large multi-modal model. In _European Conference on Computer Vision_ , pp. 74–91. Springer, 2024c. 

- Zhixiong Zhang, Shuangrui Ding, Xiaoyi Dong, Songxin He, Jianfan Lin, Junsong Tang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Sec: Advancing complex video object segmentation via progressive concept construction. _arXiv preprint arXiv:2507.15852_ , 2025.


key: chroma:document
string_value:
- Shiyu Zhao, Long Zhao, Yumin Suh, Dimitris N Metaxas, Manmohan Chandraker, Samuel Schulter, et al. Generating enhanced negatives for training language-based object detectors. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 13592–13602, 2024. 

- Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ade20k dataset. _International Journal of Computer Vision_ , 127(3):302–321, 2019. 

- Yang Zhou, Shiyu Zhao, Yuxiao Chen, Zhenting Wang, Can Jin, and Dimitris N Metaxas. Led: Llm enhanced open-vocabulary object detection without human curated data generation. _arXiv preprint arXiv:2503.13794_ , 2025. 

- Pengfei Zhu, Longyin Wen, Dawei Du, Xiao Bian, Heng Fan, Qinghua Hu, and Haibin Ling. Detection and tracking meet drones challenge. _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 44(11):7380–7399, 2021.


key: chroma:document
string_value:
- Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. _arXiv preprint arXiv:2010.04159_ , 2020. 

- Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once. _Advances in neural information processing systems_ , 36:19769–19782, 2023. 

--- end of page.page_number=76 ---


key: chroma:document
string_value:
**Data Analysis** 

## **DeepAnalyze: Agentic Large Language Models for Autonomous Data Science** 

**Data Preparation** 

**Shaolei Zhang**[1] **Ju Fan**[1] **Meihao Fan**[1] **Guoliang Li**[2] **Xiaoyong Du**[1] 

� ruc-datalab/DeepAnalyze DeepAnalyze-8B DataScience-Instruct-500K ruc-deepanalyze.github.io 

## **Abstract**


key: chroma:document
string_value:
Autonomous data science, from raw data sources to analyst-grade deep research reports, has been a long-standing challenge, and is now becoming feasible with the emergence of powerful large language models (LLMs). Recent workflow-based data agents have shown promising results on specific data tasks but remain fundamentally limited in achieving fully autonomous data science due to their reliance on predefined workflows. In this paper, we introduce _DeepAnalyze-8B_ , the first agentic LLM designed for autonomous data science, capable of automatically completing the end-toend pipeline from data sources to analyst-grade deep research reports. To tackle high-complexity data science tasks, we propose a curriculum-based agentic training paradigm that emulates the learning trajectory of human data scientists, enabling LLMs to progressively acquire and integrate multiple capabilities in real-world environments. We also introduce a data-grounded trajectory synthesis framework that constructs


key: chroma:document
string_value:
scientists, enabling LLMs to progressively acquire and integrate multiple capabilities in real-world environments. We also introduce a data-grounded trajectory synthesis framework that constructs high-quality training data. Through agentic training, _DeepAnalyze_ learns to perform a broad spectrum of data tasks, ranging from data question answering and specialized analytical tasks to open-ended data research. Experiments demonstrate that, with only 8B parameters, DeepAnalyze outperforms previous workflow-based agents built on most advanced proprietary LLMs. The model, code, and training data of DeepAnalyze are open-sourced, paving the way toward autonomous data science.


key: chroma:document
string_value:
## **1. Introduction** 

Autonomous data science (De Bie et al., 2022; Sun et al., 2025b; Wang et al., 2025), a long-standing central goal of the data science community, aims to automate the _entire data_ 

> 1Renmin University of China 2Tsinghua University. Email: Shaolei Zhang <zhangshaolei98@ruc.edu.cn>, Ju Fan (corresponding author) <fanj@ruc.edu.cn>. 

_Preprint._ 

**==> picture [236 x 104] intentionally omitted <==**


key: chroma:document
string_value:
Data Preparation<br>action Data Analysis<br>Data Modeling<br>Data Source Environment Orchestration Data Visualization<br>Data Insight<br>Research on the data, find some insight! feedback reasoning 1,000 fee rules across multiple dimensions. The study reveals intricate relationships between merchant characteristics,This comprehensive analysis examines the complex pricing architecture of payment processing systems, analyzingThe dataset encompasses 769 merchant category codes, 4 card schemes, and 6 authorization characteristics, creatingThe analysis reveals significant variation across pricing parameters:The merchant category codes demonstrate the highest diversity with 49 distinct values, reflecting the wide rangeThese correlations indicate that certain merchant types and transaction types consistently combine with specificThe fee rules demonstrate substantial complexity: Comprehensive Analysis of Payment Processing Fee StructuresExecutive Summary transaction attributes, and fee


key: chroma:document
string_value:
types consistently combine with specificThe fee rules demonstrate substantial complexity: Comprehensive Analysis of Payment Processing Fee StructuresExecutive Summary transaction attributes, and fee structures that determine payment processing costs. Through rigorous statisticalanalysis and visualization, we uncover key patterns that explain how payment processors optimize pricing whilebalancing risk and service quality. Methodology Our analysis employed a multi-method approach combining: - Descriptive statistics of 1,000 fee rules - Correlationanalysis of 6 key parameters - Rule complexity scoring - Parameter distribution visualization - Fee simulationmodelinga complex pricing matrix that adapts to over 49 distinct merchant categories. Key FindingsParameter Diversity and InteractionsParameter Diversity Metrics: of business types served by payment processors. Authorization Characteristics Indicators (ACI) show moderatediversity with 6 distinct values that categorize transaction types.


key: chroma:document
string_value:
Diversity Metrics: of business types served by payment processors. Authorization Characteristics Indicators (ACI) show moderatediversity with 6 distinct values that categorize transaction types. Card schemes show moderate diversity with 4distinct payment networks. Parameter Correlations: payment methods, creating distinct pricing segments. Rule Complexity AnalysisRule Complexity Statistics: 1. Account Type & Credit Transactions (r=327)2. Card Scheme & Account Type (r=196)3. Card Scheme & Credit Transactions (r=132)Analysis of fee components reveals:The average fee rule incorporates elements from 8.7 different parameters, with some rules incorporating up to 15ThThe fee structure combines fixed and variable components, creating predictable baseline costs with variable compo-This analysis reveals a sophisticated pricing ecosystem where multiple parameters interact to create tailored pricingdistinct characteristics. This complexity enables precise pricing but creates challenges for


key: chroma:document
string_value:
reveals a sophisticated pricing ecosystem where multiple parameters interact to create tailored pricingdistinct characteristics. This complexity enables precise pricing but creates challenges for merchants navigating thepricing architecture. Fe fee = fixed_amount + (rate * transaction_value / 10,000) Fee Component Distribution: nents tied to transaction value. This structure allows processors to balance risk and service quality while maintainingpricing flexibility. Business ImplicationsFor Payment ProcessorsFor MerchantsConclusion for diverse merchant needs. The complex interplay between merchant characteristics, transaction attributes, and• Fixed Amount Range: €0.01-€0.21 • Rate Range: 1-190 basis points • Average Fee for €100 Transaction: €1.421.2.3.1.2.3.e fee calculation follows the formula: The strongest parameter correlations reveal important pricing interactions: e Structure Mechanics ParameterMerchant Category CodeAuthorization Characteristics IndicatorCard SchemeCredit


key: chroma:document
string_value:
the formula: The strongest parameter correlations reveal important pricing interactions: e Structure Mechanics ParameterMerchant Category CodeAuthorization Characteristics IndicatorCard SchemeCredit TransactionsIntracountry StatusAccount Type  Pricing Architecture Rule Simplification Parameter Interactions Volume Optimization Fraud Management Parameter Selection can improve risk-adjusted pricing.solidation without significant revenue impact.segments for targeted pricing.tion.impacts costs. This comprehensive analysis provides both strategic insights for payment processors and practical guidance forpayment methods creates both challenges and opportunities for all stakeholders in the payment processing ecosystem.Future research cosegments and economic conditions. AppendicesAppendix A: Parameter Distribution VisualizationsAppendix B: Complete Parameter Correlation MatrixAppendix C: Fee Simulation Examples Detailed examples of fee calculations across parameter combinationsmerchants


key: chroma:document
string_value:
Parameter Distribution VisualizationsAppendix B: Complete Parameter Correlation MatrixAppendix C: Fee Simulation Examples Detailed examples of fee calculations across parameter combinationsmerchants navigating complex payment pricing architectures.: The high complexity (avg. 8.7 parameters/rule) suggests opportunities for rule con-: Strong correlations between fraud levels and pricing suggest fraud prevention directly: Understanding parameter interactions can help optimize pricing strategies.: The complex parameter interactions create natural segmentation opportunities that: The volume-based pricing curves create clear incentives for transaction consolida-1: Strong correlations between account types and payment methods indicate naturalFigure 1: Parameter distribution charts showing value frequencies across key parameters uld explore longitudinal ana MetricMean ComplexityStandard DeviationMinimum25th PercentileMedian75th PercentileMaximum6Diversity Score495420.25 Figure 2: Full


key: chroma:document
string_value:
value frequencies across key parameters uld explore longitudinal ana MetricMean ComplexityStandard DeviationMinimum25th PercentileMedian75th PercentileMaximum6Diversity Score495420.25 Figure 2: Full correlation matrix of all parameter pairs2 Value8.722.820891015 lysis of how these parameters interact across different merchant3<br>Research  Analyst-grade<br>Goal Autonomous Data Science Report<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
_Figure 1._ DeepAnalyze-8B is the first end-to-end agentic LLM that achieves autonomous data science, supporting entire data science pipeline and open-ended data research.


key: chroma:document
string_value:
_science pipeline_ for extracting insights from structured data. This pipeline is inherently complex, consisting of a series of interdependent data-centric tasks spanning data preparation, analysis, modeling, visualization, and report generation. The emergence of open-ended data research further elevates the level of complexity, going far beyond traditional question answering or task-specific analytics. Fortunately, recent advances in large language models (LLMs) have demonstrated impressive problem-solving abilities (OpenAI, 2023; 2024; DeepSeek-AI, 2025), reshaping paradigms in domains such as search (Zheng et al., 2025; Jin et al., 2025) and mathematics (Zhang et al., 2024; Ren et al., 2025). However, despite their success on unstructured data (e.g., textual queries or contexts), LLMs still struggle to orchestrate complex, multi-stage data science pipelines and handle diverse structured data, making it difficult to achieve a general solution that works across all data science


key: chroma:document
string_value:
LLMs still struggle to orchestrate complex, multi-stage data science pipelines and handle diverse structured data, making it difficult to achieve a general solution that works across all data science tasks.


key: chroma:document
string_value:
Addressing these challenges requires endowing LLMs with two higher-level capabilities: autonomous orchestration and adaptive optimization. First, autonomous orchestration enables LLMs to comprehend user intents and systematically coordinate a sequence of interdependent actions to accomplish complex tasks (Sapkota et al., 2025a). Second, adaptive optimization allows LLMs to interact with real-world data environments and iteratively refine their actions based on feedback (Hong et al., 2024). As shown in Figure 4, equipped with these two capabilities, an intelligent system can robustly handle a broad spectrum of data tasks, ranging from conventional question answering and task-specific analytics to fully autonomous, open-ended data research. 

--- end of page.page_number=1 ---

**==> picture [487 x 430] intentionally omitted <==**


key: chroma:document
string_value:
DeepAnalyze: Agentic Large Language Models for Autonomous Data Science<br>End-to-end Data Pipeline Open-ended Data Research<br>User ( Case from DataSciBench, involving data preparation + data analysis + data visualization. ) User ( Case from DABStep-Research, conducting fully open research on structured data sources. )<br>Embark on a detailed analysis and visualization of the "Palmer Archipelago (Antarctica) penguin data" dataset available on Kaggle, aiming to uncover insights about penguin populations.1.    - 2.    - Address missing values through imputation or removal, encode categorical variables for analysis, and eliminate any duplicate entries.   -    - 3.    - Compute essential statistical measures including mean, median, and standard deviation for quantitative attributes, and compile summary statistics for categorical variables.   -    - 4.  Import the datasetData CleaningExploratory Data Analysis (EDA)Data VisualizationInputInputOutputInputOutput : "penguins_lter.csv" and


key: chroma:document
string_value:
summary statistics for categorical variables.   -    - 4.  Import the datasetData CleaningExploratory Data Analysis (EDA)Data VisualizationInputInputOutputInputOutput : "penguins_lter.csv" and "penguins_size.csv": "imported_data.csv": "cleaned_data.csv": "cleaned_data.csv" (This file should contain the cleaned data, which is free of duplicates and has handled missing values): "eda_results.csv" (This file should summarize the statistical measures and compositions of the dataset): :: : Environment: # Instruction Conduct open-ended data science research, finally output a research report. # Data File 1: {“name”: “acquirer_countries.csv”, "size": "0.2KB"}File 2: {“name”: “fees.json”, “size”: “518.7KB”}File 3: {“name”: “manual.md”, “size”: “21.6KB”}File 4: {“name”: “merchant_category_codes.csv”, “size”: “26.0KB”}File 5: {“name”: “merchant_data.json”, “size”: “6.7KB”}File 6: {“name”: “payments-readme.md”, “size”: “1.7KB”}File 7: {“name”: “payments.db”, "size": "21.7MB"} Environment:<br>①②③④⑤


key: chroma:document
string_value:
“size”: “26.0KB”}File 5: {“name”: “merchant_data.json”, “size”: “6.7KB”}File 6: {“name”: “payments-readme.md”, “size”: “1.7KB”}File 7: {“name”: “payments.db”, "size": "21.7MB"} Environment:<br>①②③④⑤ <Analyze> To begin this analysis, I'll first need to understand the structure and content of the provided datasets. The task involves two CSV files: "penguins_lter.csv" and "penguins_size.csv". Since we need to import these files, I'll start by writing Python code to load and inspect these datasets.Key steps for initial data inspection:This initial exploration will help me understand how to proceed with the data cleaning and subsequent analysis steps. </Analyze><Execute> === Penguins LTER Dataset Information ===Shape: (344, 17)Columns and Data Types:studyName               objectSample Number            int64Species                 object...( </Execute><Code></Code><Understand></Understand><Code>…</Code><Analyze></Analyze>    - Construct histograms for numerical attributes and bar plots


key: chroma:document
string_value:
int64Species                 object...( </Execute><Code></Code><Understand></Understand><Code>…</Code><Analyze></Analyze>    - Construct histograms for numerical attributes and bar plots for categorical variables. Use box plots to examine variations across different species and islands, and scatter plots to investigate correlations between numerical variables with regression lines.   -    - 5.    - Specifically investigate the relationship between bill length and flipper length, including a scatter plot with regression line.   -    - Based on the execution results, I can make the following observations about the datasets:1. **Dataset Structure**:   - The "penguins_lter.csv" contains 344 rows and 17 columns with detailed penguin data including species, location, physical measurements, and isotopic data.   - The "penguins_size.csv" contains 344 rows and 7 columns with more focused physical measurements (culmen, flipper length, body mass).2. **Key Differences**:   - The


key: chroma:document
string_value:
and isotopic data.   - The "penguins_size.csv" contains 344 rows and 7 columns with more focused physical measurements (culmen, flipper length, body mass).2. **Key Differences**:   - The datasets have no common columns, suggesting they contain complementary information rather than overlapping data.   - The "penguins_lter" dataset contains more detailed information including:     * Species classification (Adelie, Chinstrap, Gentoo)     * Geographic information (island, region)     * Biological measurements (culmen, flipper length, body mass)     * Isotopic data (Delta 15N, Delta 13C)     * Reproductive information (clutch completion dates)3. **Data Quality**:1.import 2.# Load both datasets 3.penguins_lter = pd.read_csv(4.penguins_size = pd.read_csv( 5.# Function to display dataset information 6.def7.8.9.10.Given the dataset characteristics, I'll proceed with the following plan:1. Data Cleaning:   - Standardize column names between datasets   - Handle missing values (imputation or


key: chroma:document
string_value:
6.def7.8.9.10.Given the dataset characteristics, I'll proceed with the following plan:1. Data Cleaning:   - Standardize column names between datasets   - Handle missing values (imputation or removal)   - Remove duplicate entries   - Merge datasets where appropriate2. Data Exploration:   - Examine distributions of key variablesmore results are omitted to save space Correlation AnalysisInputOutputInputOutput 1. Load both CSV files using pandas2. Display basic information about each dataset (shape, columns, data types)3. Check for missing values4. Display sample rows to understand the data structure5. Compare the two datasets to see if they contain similar or complementary information inspect_dataset(df, name):: "cleaned_data.csv": "cleaned_data.csv"printprintprintprint: "visualizations.pdf" (This file should include all plots and graphs created during the analysis): "correlation_analysis.png" (This file should present the scatter plot displaying the correlation between the specified


key: chroma:document
string_value:
(This file should include all plots and graphs created during the analysis): "correlation_analysis.png" (This file should present the scatter plot displaying the correlation between the specified numerical variables) pandas(((f"\n===f"Shape:"\nColumns and Data Types:"(df.dtypes)  , <Execute>…</Execute>  as: pd {name} {df.shape} Dataset Information ==="'../penguins_lter.csv''../penguins_size.csv'") )  , <Understand>…</Understand> ) actively explore each data)))  are omitted to save space... planning first11.12.13.14. 15.# Inspect both datasets 16.print17.print 18.# Check if datasets have common columns 19.common_cols =20.print   - Both datasets have missing values, particularly in:     * Body mass and isotopic measurements     * Sex information     * Comments field in the larger dataset   - The "penguins_size" dataset has fewer missing values overall.4. **Potential Analysis Paths**:   - The datasets can be merged on common identifiers (Species, Island, etc.)   - The "penguins_size"


key: chroma:document
string_value:
- The "penguins_size" dataset has fewer missing values overall.4. **Potential Analysis Paths**:   - The datasets can be merged on common identifiers (Species, Island, etc.)   - The "penguins_size" dataset appears to be a subset of the more comprehensive "penguins_lter" data   - The isotopic data in "penguins_lter" could provide interesting ecological insights5. **Next Steps**:   - Need to clean and merge the datasets   - Handle missing values appropriately   - Standardize column names for merging   - Explore relationships between physical measurements   - Compare measurements across species   - Investigate relationships between physical measurements3. Visualization:   - Create histograms for key measurements   - Generate boxplots by species   - Scatter plots for correlation analysisFirst, I‘ll write code to clean and merge the datasets: ...(printprintprintreturn(inspect_dataset(penguins_lter,(inspect_dataset(penguins_size,(f"\nCommon columns between datasets:((df.isnull().(


key: chroma:document
string_value:
I‘ll write code to clean and merge the datasets: ...(printprintprintreturn(inspect_dataset(penguins_lter,(inspect_dataset(penguins_size,(f"\nCommon columns between datasets:((df.isnull().( df.head("\nMissing Values:""\nSample Data:" set(penguins_lter.columns) &3) sumcleaned_data.csv())) )  "Penguins LTER" "Penguins Size" set {common_cols}understand the data structureomitted to save space(penguins_size.columns))))) ") ) ⑦⑥②④⑤①③ <Execute> Merchant Category Codes Data:==================================================Number of MCCs: 769First 10 MCCs:   Unnamed: 0   mcc                                        description0           0  1520   General Contractors - Residential and Commercial <Understand> The manual.md file provides comprehensive documentation about payment processing systems, including:1. Key business concepts:- Merchant account types (R, D, H, F, S, O)- Merchant Category Codes (MCC) for business classification- Authorization Characteristics Indicator (ACI) for transaction


key: chroma:document
string_value:
Key business concepts:- Merchant account types (R, D, H, F, S, O)- Merchant Category Codes (MCC) for business classification- Authorization Characteristics Indicator (ACI) for transaction types- Fee calculation methodology (fixed_amount + rate * transaction_value/10000)...(The manual provides the essential context needed to properly interpret the other data files, particularly fees.json which contains the actual fee rules </Understand><Analyze> Based on the manual content, the next logical steps are:The immediate next step should be to examine merchant_category_codes.csv since MCC is a key parameter in fee rules. This will help understand how businesses are categorized for pricing purposes.I'll generate code to: </Analyze><Analyze> To begin this comprehensive data science research, I will first need to understand the structure and content of each provided file. The files appear to relate to payment processing data, including merchant information, transaction fees, and payment records.


key: chroma:document
string_value:
first need to understand the structure and content of each provided file. The files appear to relate to payment processing data, including merchant information, transaction fees, and payment records. Here's my initial analysis plan:The immediate next step is to inspect the manual.md file which likely contains important documentation about the dataset structure and business rules. This will help guide my analysis of the other files.I'll start by writing Python code to read and display the contents of manual.md, as this will provide crucial context for interpreting the other files. </Analyze><Code></Code><Execute> Manual.md content:==================================================# Merchant Guide to Optimizing Payment Processing and Minimizing Fees...( </Execute><Code></Code> more results are omitted to save spacemore results are omitted to save space 1.# Read and display the manual.md file 2.with3.4.5.6.1.import2.import 3.# Read the merchant category codes file 4.mcc_df = pd.read_csv(


key: chroma:document
string_value:
omitted to save spacemore results are omitted to save space 1.# Read and display the manual.md file 2.with3.4.5.6.1.import2.import 3.# Read the merchant category codes file 4.mcc_df = pd.read_csv( 5.# Display basic information 6.print7.print8.print9.print10.print11.print12.print 13.# Show summary statistics 14.print15.print1. Examine the merchant_category_codes.csv to understand the MCC taxonomy used in fee calculations2. Analyze fees.json which contains the actual fee rules referenced in the manual3. Explore merchant_data.json to understand merchant attributes that influence fees4. Investigate payments.db for transaction-level data1. Read the CSV file2. Display sample records3. Show summary statistics4. Visualize the distribution of MCCs1. First, I'll examine the smallest files (acquirer_countries.csv and manual.md) to understand the basic context and data dictionary2. Then I'll analyze the merchant data (merchant_data.json and merchant_category_codes.csv)3. Next, I'll explore the


key: chroma:document
string_value:
and manual.md) to understand the basic context and data dictionary2. Then I'll analyze the merchant data (merchant_data.json and merchant_category_codes.csv)3. Next, I'll explore the fee structure (fees.json) which appears to be the largest file4. Finally, I'll examine the payment transaction data (payments.db and payments-readme.md)content = file.read()printprintprint open(((( pandas matplotlib.pyplot"Merchant Category Codes Data:""="f"Number of MCCs:"\nFirst 10 MCCs:"(mcc_df.head(((mcc_df.tail((("\nLast 10 MCCs:""\nSummary Statistics:""="((((content)*'manual.md'"Manual.md content:""="50*50)* as)50) pd1010,'merchant_category_codes.csv')))) 'r' { as))len) as plt(mcc_df)} file:)))) )") ) explore the overview of data sourcesidentify key data (manual.md) and 16.print 17.# Visualize the distribution of MCCs by first two digits (category) 18.mcc_df[19.category_counts =


key: chroma:document
string_value:
file:)))) )") ) explore the overview of data sourcesidentify key data (manual.md) and 16.print 17.# Visualize the distribution of MCCs by first two digits (category) 18.mcc_df[19.category_counts = mcc_df[20.plt.figure(figsize=(21.category_counts.plot(kind=22.plt.title(23.plt.xlabel(24.plt.ylabel(25.plt.xticks(rotation=26.plt.tight_layout()27.plt.savefig(28.print.png'"7.8.9.10.11.12. )print # Also save the content to a text file for reference withprint(mcc_df.describe(include=("\nSaved MCC category distribution plot to 'mcc_category_distribution'category'explore the database (payments.db) andtables (merchant_category_codes.csv )(outfile.write(content) open"="('Distribution of Merchant Category Codes by First Two Digits'"\nSaved manual content to 'manual_content.txt'"'Category Code (First 2 Digits)''Number of MCCs''mcc_category_distribution.png'*50('manual_content.txt') ] = mcc_df[1245,) 6planning first'category'))'bar')'mcc''all') ].astype(,))].value_counts().sort_index() 'w' . )


key: chroma:document
string_value:
2 Digits)''Number of MCCs''mcc_category_distribution.png'*50('manual_content.txt') ] = mcc_df[1245,) 6planning first'category'))'bar')'mcc''all') ].astype(,))].value_counts().sort_index() 'w' . ) asstr)) outfile:).str[:2] ) )<br>⑥ <Analyze> The cleaned data is now ready for in-depth analysis. For the next steps, I'll:1. Perform comprehensive EDA:   - Calculate descriptive statistics   - Analyze distributions by species   - Examine measurement correlations2. Create visualizations: 3. Conduct correlation analysis:   - Focus on bill length vs flipper length   - Generate regression plots4. Save all outputs:   - Statistical summaries   - Visualization files data preparation Several  1           1  1711  Heating, Plumbing, and Air Conditioning Contra...2           2  1731                             Electrical Contractors...(Saved MCC category distribution plot to 'mcc_category_distribution.png’ </Execute> more results are omitted to save space <Understand>…</Understand>, )


key: chroma:document
string_value:
Electrical Contractors...(Saved MCC category distribution plot to 'mcc_category_distribution.png’ </Execute> more results are omitted to save space <Understand>…</Understand>, )  <Analyze>…</Analyze>, <Code>…</Code> , <Execute>…</Execute> are omitted to save space..<br>Several</Analyze>    - Boxplots by species   - Scatter plots for correlations   - Histograms for measurement distributions  <Code>…</Code> , <Execute>…</Execute> , <Understand>…</Understand> are omitted to save space...    - Final analysis reportThe code will generate all required outputs while maintaining clear documentation. ⑧ <Answer> # Comprehensive Analysis of Payment Processing Fee Structures## Executive SummaryThis comprehensive analysis examines the complex pricing architecture of payment processing systems, analyzing 1,000 fee rules across multiple dimensions. The study reveals intricate relationships between merchant characteristics, transaction attributes, and fee structures that


key: chroma:document
string_value:
processing systems, analyzing 1,000 fee rules across multiple dimensions. The study reveals intricate relationships between merchant characteristics, transaction attributes, and fee structures that determine payment processing costs. Through rigorous statistical analysis and visualization, we uncover key patterns that explain how payment processors optimize pricing while balancing risk and service quality....(more results are omitted to save spaceresearch on the data)<br>eda_results.csv culmen_vs_ correlation body_mass </Answer> data science research report<br>⑦ <Answer></Answer><Understand>…</Understand> # Comprehensive Analysis of Palmer Archipelago Penguin Data## 1. Data Preparation and Cleaning- Successfully merged two datasets containing penguin measurements- Handled missing values by removing incomplete records- Final cleaned dataset contains 344 complete observations## 2. Key Findings### Physical Characteristics by Species- **Adelie Penguins**:  - Average body mass: 3,750g  -


key: chroma:document
string_value:
by removing incomplete records- Final cleaned dataset contains 344 complete observations## 2. Key Findings### Physical Characteristics by Species- **Adelie Penguins**:  - Average body mass: 3,750g  - Average culmen length: 39.1mm- **Chinstrap Penguins**:  - Distinctive culmen depth (18.7mm)-  - Largest body mass (5,000g average)### Measurement Correlations- Strong positive correlation between bill length and flipper length (r=0.82)- Significant variation in body mass across species (p<0.001)## 3. Visualizations Generated1. **Body Mass Distribution**:   - Clear species differentiation in mass distributions2. **Bill vs Flipper Length**:**Gentoo Penguins**:data analysis , <Analyze>…</Analyze>, <Code>…</Code> flipper  , <Execute>…</Execute>    - Linear relationship with positive slope3. **Geographic Variation**:   - Island-specific measurement patterns## 4. Biological Insights- Species-specific morphological adaptations- Island-specific size variations- Sex-based differences in body


key: chroma:document
string_value:
**Geographic Variation**:   - Island-specific measurement patterns## 4. Biological Insights- Species-specific morphological adaptations- Island-specific size variations- Sex-based differences in body proportions## 5. Methodology- Rigorous data cleaning process- Multiple validation checks- Comprehensive statistical testing## 6. ConclusionThis analysis provides valuable insights into penguin morphology and ecology, demonstrating:- Species-specific adaptations- Environmental influences- Measurement relationshipsAll analysis files and visualizations are available in the output directory.![Body Mass by Species](body_mass_by_species.png)![Bill Length vs Flipper Length](correlation_analysis.png)  are omitted to save space.. _analysis final summarization_by_speciesdata visualization 1,000 fee rules across multiple dimensions. The study reveals intricate relationships between merchant characteristics,This comprehensive analysis examines the complex pricing architecture of payment processing


key: chroma:document
string_value:
across multiple dimensions. The study reveals intricate relationships between merchant characteristics,This comprehensive analysis examines the complex pricing architecture of payment processing systems, analyzingThe dataset encompasses 769 merchant category codes, 4 card schemes, and 6 authorization characteristics, creatingThe analysis reveals significant variation across pricing parameters:The merchant category codes demonstrate the highest diversity with 49 distinct values, reflecting the wide rangeThese correlations indicate that certain merchant types and transaction types consistently combine with specificThe fee rules demonstrate substantial complexity: Comprehensive Analysis of Payment Processing Fee StructuresExecutive Summary transaction attributes, and fee structures that determine payment processing costs. Through rigorous statisticalanalysis and visualization, we uncover key patterns that explain how payment processors optimize pricing whilebalancing risk and service


key: chroma:document
string_value:
payment processing costs. Through rigorous statisticalanalysis and visualization, we uncover key patterns that explain how payment processors optimize pricing whilebalancing risk and service quality. Methodology Our analysis employed a multi-method approach combining: - Descriptive statistics of 1,000 fee rules - Correlationanalysis of 6 key parameters - Rule complexity scoring - Parameter distribution visualization - Fee simulationmodelinga complex pricing matrix that adapts to over 49 distinct merchant categories. Key FindingsParameter Diversity and InteractionsParameter Diversity Metrics: of business types served by payment processors. Authorization Characteristics Indicators (ACI) show moderatediversity with 6 distinct values that categorize transaction types. Card schemes show moderate diversity with 4distinct payment networks. Parameter Correlations: payment methods, creating distinct pricing segments. Rule Complexity AnalysisRule Complexity Statistics: 1. Account Type & Credit


key: chroma:document
string_value:
diversity with 4distinct payment networks. Parameter Correlations: payment methods, creating distinct pricing segments. Rule Complexity AnalysisRule Complexity Statistics: 1. Account Type & Credit Transactions (r=327)2. Card Scheme & Account Type (r=196)3. Card Scheme & Credit Transactions (r=132) The strongest parameter correlations reveal important pricing interactions:ParameterMerchant Category CodeAuthorization Characteristics IndicatorCard SchemeCredit TransactionsIntracountry StatusAccount Type 1 6Diversity Score495420.25 Analysis of fee components reveals:The average fee rule incorporates elements from 8.7 different parameters, with some rules incorporating up to 15The fee calculation follows the formula:The fee structure combines fixed and variable components, creating predictable baseline costs with variable compo-This analysis reveals a sophisticated pricing ecosystem where multiple parameters interact to create tailored pricingdistinct characteristics. This complexity


key: chroma:document
string_value:
baseline costs with variable compo-This analysis reveals a sophisticated pricing ecosystem where multiple parameters interact to create tailored pricingdistinct characteristics. This complexity enables precise pricing but creates challenges for merchants navigating thepricing architecture. Fee Structure Mechanics fee = fixed_amount + (rate * transaction_value / 10,000) Fee Component Distribution: nents tied to transaction value. This structure allows processors to balance risk and service quality while maintainingpricing flexibility. Business ImplicationsFor Payment ProcessorsFor MerchantsConclusion for diverse merchant needs. The complex interplay between merchant characteristics, transaction attributes, and• Fixed Amount Range: €0.01-€0.21• Rate Range: 1-190 basis points• Average Fee for €100 Transaction: €1.421.2.3.1.2.3.  Pricing Architecture Rule Simplification Parameter Interactions Volume Optimization Fraud Management Parameter Selection can improve risk-adjusted


key: chroma:document
string_value:
Average Fee for €100 Transaction: €1.421.2.3.1.2.3.  Pricing Architecture Rule Simplification Parameter Interactions Volume Optimization Fraud Management Parameter Selection can improve risk-adjusted pricing.solidation without significant revenue impact.segments for targeted pricing.tion.impacts costs. : The high complexity (avg. 8.7 parameters/rule) suggests opportunities for rule con-: Strong correlations between fraud levels and pricing suggest fraud prevention directly: Understanding parameter interactions can help optimize pricing strategies.: The complex parameter interactions create natural segmentation opportunities that: The volume-based pricing curves create clear incentives for transaction consolida-: Strong correlations between account types and payment methods indicate naturalMetricMean ComplexityStandard DeviationMinimum25th PercentileMedian75th PercentileMaximum 2 8.722.82Value0891015 This comprehensive analysis provides both strategic insights for payment processors


key: chroma:document
string_value:
ComplexityStandard DeviationMinimum25th PercentileMedian75th PercentileMaximum 2 8.722.82Value0891015 This comprehensive analysis provides both strategic insights for payment processors and practical guidance forpayment methods creates both challenges and opportunities for all stakeholders in the payment processing ecosystem.Future research could explore longitudinal analysis of how these parameters interact across different merchantsegments and economic conditions. AppendicesAppendix A: Parameter Distribution VisualizationsAppendix B: Complete Parameter Correlation MatrixAppendix C: Fee Simulation Examples Detailed examples of fee calculations across parameter combinationsmerchants navigating complex payment pricing architectures.Figure 1: Parameter distribution charts showing value frequencies across key parametersFigure 2: Full correlation matrix of all parameter pairs3<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
_Figure 2._ Examples of DeepAnalyze-8B. Given the instructions and data sources in the environment, DeepAnalyze can autonomously orchestrate and optimize actions to complete a data science pipeline (left) and open-ended data research (right). DeepAnalyze first performs planning, then interacts with the data in the environment, and subsequently optimizes its actions based on feedback, ultimately accomplishing the data-centric tasks. Many intermediate actions are omitted to save space.


key: chroma:document
string_value:
Existing approaches to applying LLMs for autonomous data science can be broadly categorized into _domain-specific LLMs_ and _workflow-based agents_ . Early efforts focus on developing domain-specific LLMs, such as code-oriented models (Nascimento et al., 2024; Wen et al., 2024) and structured data-oriented models (Li et al., 2023b; Jiang et al., 2023; Xu et al., 2025), to handle individual tasks like question answering or specific analytical operations. However, these models lack the capabilities for autonomous orchestration and adaptive optimization (Yang et al., 2021; Li et al., 2023a), limiting their ability to execute the entire data science pipeline. More recently, a line of work has explored workflow-based data science agents (Hollmann et al., 2023; Guo et al., 2024; Sun et al., 2025a; Hong et al., 2025),


key: chroma:document
string_value:
which rely on predefined procedural workflows to prompt closed-source LLMs (e.g., GPT-4 (OpenAI, 2023)) to complete complex tasks. Although these systems demonstrate stronger task coordination, they depend heavily on manually designed heuristics and domain-specific rules, falling short of achieving autonomous and adaptive behavior. 

In essence, both domain-specific models and workflowbased agents remain limited, as they are not trained in interactive, real-world environments. Consequently, they struggle to perform complex tasks through autonomous orchestration and adaptive optimization. Notably, recent advances in agentic training, a new training paradigm successfully applied in the search domain (Zheng et al., 2025; Jin et al., 2025), have demonstrated that reinforcement learning in 

--- end of page.page_number=2 ---

real-world environments is crucial for enabling LLMs to develop autonomous problem-solving capabilities.


key: chroma:document
string_value:
In this paper, we aim to advance LLM-based data science methods from workflow-based agents to a trainable agentic model that learns to autonomously perform data science tasks in real-world environments. However, applying agentic training to this domain presents two key challenges: _reward sparsity_ and _trajectory scarcity_ . On the one hand, the inherent complexity of data science tasks makes it difficult for foundation LLMs to complete tasks successfully during the early stages of training. This leads to severe reward sparsity, i.e., a lack of positive reinforcement signals, which can hinder or even collapse the entire agentic training process. On the other hand, the scarcity of longchain problem-solving trajectories in data science provides insufficient guidance for LLMs to explore the solution space effectively, resulting in inefficient, blind trial-and-error exploration without meaningful intermediate supervision.


key: chroma:document
string_value:
To address these challenges, we introduce DeepAnalyze, an agentic LLM designed for autonomous data science. As illustrated in Figure 2, with only 8B parameters, DeepAnalyze can automate the entire data science pipeline, ranging from specific data tasks to open-ended data research, providing a unified and general solution for data-centric applications. Specifically, to mitigate reward sparsity, DeepAnalyze adopts a curriculum-based agentic training paradigm inspired by the learning trajectory of human data scientists. This progressive easy-to-difficult schedule enables the model to gradually evolve from mastering individual skills to developing comprehensive, adaptive problem-solving abilities in real-world environments. To address trajectory scarcity, we propose a data-grounded trajectory synthesis framework that automatically constructs high-quality reasoning and interaction trajectories, offering effective exploration guidance within the large solution space. Through this


key: chroma:document
string_value:
trajectory synthesis framework that automatically constructs high-quality reasoning and interaction trajectories, offering effective exploration guidance within the large solution space. Through this combination of curriculum-based training and trajectory synthesis, DeepAnalyze learns to autonomously orchestrate actions and adaptively optimize its strategies, enabling it to tackle complex and diverse data science tasks effectively.


key: chroma:document
string_value:
In summary, our key contributions are three-fold. 

- **Agentic Model:** To the best of our knowledge, DeepAnalyze is the first agentic LLM tailored for autonomous data science, endowed with two indispensable capabilities, _autonomous orchestration_ and _adaptive optimization_ . DeepAnalyze serves as a _foundation model_ that can be directly applied or further customized through prompting or supervised fine-tuning for specific scenarios. 

- **Agentic Training** : We propose a curriculum-based agentic training paradigm with data-grounded trajectory synthesis to address _reward sparsity_ and _trajectory scarcity_ , enabling effective learning for high-complexity tasks that require multiple abilities.


key: chroma:document
string_value:
- **Strong Performance** : Experimental results on 12 benchmarks show that, with only 8B parameters, **DeepAnalyze8B** surpasses most advanced proprietary LLMs. More importantly, it is the first agentic model capable of performing open-ended data research and generating analystgrade reports. 

## **2. Related Work**


key: chroma:document
string_value:
**Autonomous Data Science.** Autonomous data science has long been pursued as an important goal of intelligent systems. Existing LLM-based data science methods can be categorized into: domain-specific LLMs and workflow-based agents. To handle individual tasks in data science, early methods focused on fine-tuning LLMs into domain-specific models, including LLMs for data science code generation (Nascimento et al., 2024; Wen et al., 2024; Nejjar et al., 2024; Pan et al., 2025), tabular LLMs (Li et al., 2023b; Fang et al., 2024; Zhang et al., 2025c; Xu et al., 2025; Ouyang et al., 2025; Lei et al., 2025), and database-oriented LLMs (Xue et al., 2024; Liu et al., 2024; Mohammadjafari et al., 2025). Recently, an increasing number of data agents have demonstrated promising performance in data science by leveraging workflows to gradually prompt LLMs for complex tasks (Hollmann et al., 2023; Guo et al., 2024; Yang et al., 2024; Sun et al., 2025a; Hong et al., 2025). Most existing agents are


key: chroma:document
string_value:
science by leveraging workflows to gradually prompt LLMs for complex tasks (Hollmann et al., 2023; Guo et al., 2024; Yang et al., 2024; Sun et al., 2025a; Hong et al., 2025). Most existing agents are built upon Chain-of-Thought frameworks, including ReAct (Yao et al., 2023), AutoGen (Wu et al., 2024), and self-reflection (Pan et al., 2023), which decompose complex tasks into multiple subtasks and solve them sequentially. Regardless of workflow design, existing agents primarily rely on carefully crafted prompting to guide closed-source LLMs in performing data science tasks.


key: chroma:document
string_value:
Despite these advances, domain-specific LLMs (focused on individual tasks) and workflow-based agents (dependent on manually designed workflows) remain incapable of fully autonomous data science. Therefore, the proposed DeepAnalyze does not rely on prompting frameworks or predefined workflows, instead, it internalizes data science capabilities through agentic training within real-world environments.


key: chroma:document
string_value:
**Agentic Training for LLM.** Agentic training aims to enhance LLMs as agentic models through reinforcement learning and thereby enable LLMs to perform multi-step reasoning and interactions in real-world environments (Plaat et al., 2025), which has already achieved practical success in coding (Sapkota et al., 2025b) and searching (Zheng et al., 2025; Li et al., 2025; Jin et al., 2025). Typically, these methods use prompts to control the interaction format of LLMs and complete RL with the accuracy of the final answer as the reward. Based on this, lightweight cold-start is proposed to help LLMs learn the interaction format (DeepSeek-AI, 2025), improving the initial state for RL training. Existing training methods mainly focus on reasoning ability, while 

--- end of page.page_number=3 ---

**==> picture [484 x 255] intentionally omitted <==**


key: chroma:document
string_value:
Data Science Tasks<br>Data Preparation  Data Analysis Data Modeling Visualization Data Insight Report Generation<br>Learning Path of Human Versatile  Data:<br>Basic  Analyze Understand Data-centric Instruction:<br>Abilities Code Capabilities Give me a comprehensive data analysis report based on these files.Requirements: (1)…(2)…<br>Beginner Practitioner Data Scientist Response:<Analyze> To complete this task, I need to (1) load the data,  Intermediate Results<br>(2) observe its structure, (3) select a statistical<br>Curriculum-based Agentic Training of DeepAnalyze </Analyze> model, … (8) finally generate an analysis report.Next, I will first look at the data. Processed Data<br><Code><br>Single-ability Data-centric Reasoning Fine-tuning Multi-ability Data-orientedPipeline Agentic RL ```python importdf = pd.read_csv(print(``` "=== First 5 Rows ===\n" pandas as pd 'data.csv’)  , df.head()) …  Statistical Analysis<br>Structural Data Open-ended </Code><br>Understanding Data Research


key: chroma:document
string_value:
```python importdf = pd.read_csv(print(``` "=== First 5 Rows ===\n" pandas as pd 'data.csv’)  , df.head()) …  Statistical Analysis<br>Structural Data Open-ended </Code><br>Understanding Data Research <Execute><br>LLM GenerationCode Data-orientedLLM Report Generation === First 5 Rows ===Name AgeBob 23 Salary6,000 DepartmentMarketing … Data Insight<br>… … </Execute><br><Understand><br>Through the execution results, I know the csv file<br>is about …, it has 4 columns… Visualization<br>AgenticTrajectory  Trajectory SynthesisReasoning Distillation Refinement Trajectory SynthesisInteraction  Questioner Files InstructionCode Solver ExecuteUnderstandAnalyze </ Understand ><Analyze><Answer> …<Understand>…<Code><Execute>…<br>Synthesis Existing Datasets SOTA LLMs Data in Environment Environment </Answer> # Analysis Report of Data:… Data Modeling<br>Real-world Shell : Python Pandas SQL Pytorch Scikit-learn Matplotlib Seaborn …<br>Environments Files : Data1.txt Data2.csv Data3.xlsx Data4.db


key: chroma:document
string_value:
</Answer> # Analysis Report of Data:… Data Modeling<br>Real-world Shell : Python Pandas SQL Pytorch Scikit-learn Matplotlib Seaborn …<br>Environments Files : Data1.txt Data2.csv Data3.xlsx Data4.db Data5.sqlite Data6.json Data7.md …<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
_Figure 3._ Architecture of DeepAnalyze. 

data science requires a broader range of abilities, such as reasoning, structured data understanding, and code generation. This complexity makes that initial LLMs (even after coldstart format learning) are generally incapable of completing complex data science tasks, leading to challanges of reward sparsity and trajectory scarcity. To this end, we propose a curriculum-based agentic training that enables LLMs to gradually acquire complex data science skills through a progression from single to multiple abilities, while employing data-grounded trajectory synthesis to generate high-quality reasoning and interaction trajectories for training. 

## **3. DeepAnalyze**


key: chroma:document
string_value:
## **3. DeepAnalyze** 

In this paper, we introduce DeepAnalyze, an agentic large language model for autonomous data science. To endow the LLM with the capability for autonomous orchestration and adaptive optimization in real-world environments, we propose a curriculum-based agentic training and data-grounded trajectory synthesis framework tailored for complex tasks with multiple abilities. Specifically, inspired by the behavior of human data scientists, we first define a set of actions that enable DeepAnalyze to directly interact with the data in its environment. Building on this architecture, we automatically synthesize high-quality data science trajectories and introduce a curriculum-based agentic training paradigm that guides DeepAnalyze through a progression from a beginner to data scientist, thereby empowering DeepAnalyze to tackle a wide spectrum of data science tasks. The architecture, curriculum-based agentic training, and data-grounded


key: chroma:document
string_value:
trajectory synthesis are introduced as follows. 

## **3.1. Architecture** 

Unlike foundation LLMs that focus on understanding and generating natural language, LLMs for data science meet the additional challenge of understanding and interaction with structured data, which is typically stored in external files. Therefore, DeepAnalyze extends natural language interaction by introducing data-oriented interaction pattern, thereby enabling LLMs to autonomously interact with realworld environments.


key: chroma:document
string_value:
**Inputs Format.** Previous structured data–specific LLMs (Li et al., 2023b; Fang et al., 2024; Zhang et al., 2025c; Xu et al., 2025; Lei et al., 2025) often converted tables stored in databases, CSV, or XLSX files into unstructured Markdown text and fed them into the LLM’s context to enable structured data understanding. However, due to context length limitations, these methods can only handle small-scale data (e.g., very small tables). When human data scientists work with large-scale data, they do not passively read and memorize every record. Instead, they actively explore each data source as needed and then plan the following steps accordingly. To this end, DeepAnalyze integrates both modes: it passively accepts structured data expressed as text in the input, while also actively inspecting external data sources according to user inputs, where the filenames of the external data sources are specified in inputs, as shown in Figure 2.


key: chroma:document
string_value:
**Interaction Pattern.** Given an instruction and the data sources in the environment, data scientists typically analyze, 

--- end of page.page_number=4 ---

**==> picture [486 x 179] intentionally omitted <==**


key: chroma:document
string_value:
Algorithm 1  Inference of DeepAnalyze Task Environments Reward 𝑟! 𝐴!<br>1:2:  Input: Output: M  Instruction Response  Q A , Environment (with interaction process)  Env , DeepAnalyze model Question-AnswerData-centric TaskOpen Research Policy LLM LLM-as-Judgement Rules AccInterpretabilityReadabilityCode AccUsefulness543210Interaction TurnsSoundnessRichness ...𝑟𝑟"# CompuGrouptation 𝐴𝐴..."#<br>3:  Initialization:  A  =  ∅ Cold Start Reference Model 𝑟$ 𝐴$<br>4:  while ⟨ Answer ⟩ · · · ⟨ /Answer ⟩ not in  A  do KL<br>5: y ←M ( Q, A ) // generate next action based on the<br>instruction  Q  and current response  A Figure 4.  Schematic diagram of agentic RL.<br>6: A ← A  +  y<br>7: if ⟨ Code ⟩ · · · ⟨ /Code ⟩ in  y  then<br>8: code ← extract_code( y )<br>9: feedback ← Env .execute(code) // interaction with ties needed for entire data science pipeline and open-ended<br>the data in the environment research. The complexity of these capabilities results in<br>10: A ← A + ⟨ Execute ⟩ + feedback + ⟨


key: chroma:document
string_value:
with ties needed for entire data science pipeline and open-ended<br>the data in the environment research. The complexity of these capabilities results in<br>10: A ← A + ⟨ Execute ⟩ + feedback + ⟨ /Execute ⟩ the limited proficiency of foundation LLMs in data science<br>11: end if domains (Zhang et al., 2025b), leading to severe rewardZhang et al., 2025b), leading to severe reward, 2025b), leading to severe reward 2025b), leading to severe reward), leading to severe reward<br>12:  end while sparsity on complex tasks and rendering existing agentic<br>13:  Return  A<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
ties needed for entire data science pipeline and open-ended research. The complexity of these capabilities results in the limited proficiency of foundation LLMs in data science domains (Zhang et al., 2025b), leading to severe rewardZhang et al., 2025b), leading to severe reward, 2025b), leading to severe reward 2025b), leading to severe reward), leading to severe reward sparsity on complex tasks and rendering existing agentic training (such as RL-Zero or RL with cold-start training (DeepSeek-AI, 2025)) ineffective due to the lack of positive feedback. To address this challenge, we propose curriculumbased agentic training, which emulates the learning path of human data scientists by gradually transitioning from mastering single abilities to integrating multiple abilities. This training framework consists of two stages, where stage 1 employs single-ability fine-tuning to strengthen the foundation LLM’s single ability, and stage 2 uses multi-ability agentic training to enable the LLM to


key: chroma:document
string_value:
consists of two stages, where stage 1 employs single-ability fine-tuning to strengthen the foundation LLM’s single ability, and stage 2 uses multi-ability agentic training to enable the LLM to apply multiple abilities in real-world environments to accomplish complex data science tasks.


key: chroma:document
string_value:
interact with the data in the environment, understand structured data, and iterate until the instruction is completed. To emulate this process, DeepAnalyze introduces five actions to automatically accomplish the data science task, including: 

- _⟨_ **Analyze** _⟩ · · · ⟨_ **/Analyze** _⟩_ : Analyze textually, including plan- 

- ning, reasoning, reflection, self-verification... 

- _⟨_ **Understand** _⟩ · · · ⟨_ **/Understand** _⟩_ : Understand the content of 

- data source, such as databases, tables, and documents. 

- _⟨_ **Code** _⟩ · · · ⟨_ **/Code** _⟩_ : Generate code to interact with the data 

- in the environment, using Python suited for data science.


key: chroma:document
string_value:
- _⟨_ **Code** _⟩ · · · ⟨_ **/Code** _⟩_ : Generate code to interact with the data 

- in the environment, using Python suited for data science. 

**Single-ability Fine-tuning.** Since most foundation LLMs have not been trained specifically for data science tasks, in this stage, we first enhance the various single abilities that data science relies, primarily including reasoning, structured data understanding, and code generation, which correspond respectively to the actions _⟨_ Analyze _⟩_ , _⟨_ Understand _⟩_ , and _⟨_ Code _⟩_ . Specifically, we fine-tune the foundation LLM using long CoT data (i.e., including reasoning traces) of general tasks, structured data understanding, code generation. This stage of training mirrors the human learning process from a beginner to a data science practitioner in acquiring specialized skills, enhancing LLM’s single ability in various aspects of data science. 

- _⟨_ **Execute** _⟩ · · · ⟨_ **/Execute** _⟩_ : Execute code and collect the feed-


key: chroma:document
string_value:
- _⟨_ **Execute** _⟩ · · · ⟨_ **/Execute** _⟩_ : Execute code and collect the feed- 

- back from the environment. 

- _⟨_ **Answer** _⟩ · · · ⟨_ **/Answer** _⟩_ : Produce the final output.


key: chroma:document
string_value:
- back from the environment. 

- _⟨_ **Answer** _⟩ · · · ⟨_ **/Answer** _⟩_ : Produce the final output. 

In practice, we extend the vocabulary of the foundation LLM to support the generation of these special tokens. During the inference, DeepAnalyze automatically switches between different actions by generating these special tokens, as shown in the right side of Figure 3. In particular, once a _⟨_ **Code** _⟩ · · · ⟨_ **/Code** _⟩_ is generated, DeepAnalyze executes the code in the environment and places the feedback in _⟨_ **Execute** _⟩ · · · ⟨_ **/Execute** _⟩_ , and then generates the next action. The detailed inference process of DeepAnalyze is shown in Algorithm 1. With this architecture, all actions (i.e., special tokens) are autonomously generated by the model without any human-defined workflows or rules, which allows DeepAnalyze to fully autonomously orchestrate and optimize each action, laying the foundation for autonomous data science.


key: chroma:document
string_value:
**Multi-ability Agentic Training.** Building on the mastery of various single abilities, we employ agentic reinforcement learning to train DeepAnalyze to apply multiple abilities in real-world environments to complete complex data science tasks. To ensure the quality of reinforcement learning, we first perform a cold start by fine-tuning the LLM on synthesized interaction trajectories, enabling it to acquire basic capabilities in orchestrating and optimizing individual actions. Subsequently, we train DeepAnalyze in real-world environments using reinforcement learning with group relative policy optimization (GRPO) (Shao et al., 2024). For each question _q_ in training data _D_ , GRPO samples a group of _G_ outputs _{o_ 1 _, · · · , oG}_ from the old policy _πθ_ old and then optimizes the policy model _πθ_ by maximizing the fol- 

## **3.2. Curriculum-based Agentic Training**


key: chroma:document
string_value:
## **3.2. Curriculum-based Agentic Training** 

Under the above architecture, DeepAnalyze need to learn how to interact with the environment to accomplish various data science tasks. Unlike individual coding or searching task, data science tasks demand a broader and more complex set of abilities, ranging from reasoning, structured data understanding, and code generation to the composite abili- 

--- end of page.page_number=5 ---

**==> picture [468 x 105] intentionally omitted <==**


key: chroma:document
string_value:
--- end of page.page_number=5 ---

**==> picture [468 x 105] intentionally omitted <==**

**----- Start of picture text -----**<br>
SOTA LLMsSOTA LLMs KK ey Reasoning Vocabulary Tasks Questioner Solver<br>Preparation<br>ButBut Let me checkLet me check Look at each row Analysis Give instruction Checklist ü Checklist üInteraction DetailsInteraction Details<br>Let’s revisit structuLet’s revisit structu red data Wait … Modeling… üüEnvironment ChangeEnvironment Change<br>Research<br>Observe environment Interaction<br>Reasoning TraceReasoning Trace Insert keInsert ke y reasoning words<br>Real-world Environment<br>DistillationDistillation Refinement Inspecto rr<br>(a) Reasoning Trajectory Synthesis (b) Interaction Trajectory Synthesis<br>**----- End of picture text -----**<br>


_Figure 5._ The proposed data-grounded trajectory synthesis for the development of DeepAnalyze on data science tasks. 

lowing objective: 

**==> picture [229 x 92] intentionally omitted <==**


key: chroma:document
string_value:
_Figure 5._ The proposed data-grounded trajectory synthesis for the development of DeepAnalyze on data science tasks. 

lowing objective: 

**==> picture [229 x 92] intentionally omitted <==**

where _Ai_ is the advantage calculated from the rewards _{r_ 1 _, · · · , rG}_ of outputs within each group, _π_ ref is the reference model, _ε_ and _β_ are hyperparameters.


key: chroma:document
string_value:
**Hybrid Reward Modeling.** The effectiveness of agentic reinforcement learning critically depends on both the training data and the reward function. We use the agentic interaction trajectories synthesized in Section 3.3 as training data, covering three broad categories of data science tasks: data question answering, specific data tasks (e.g., data preparation, analysis, visualization, modeling, and insight extraction), and open-ended research. Since many data science tasks are inherently open-ended, we adopt a hybrid reward modeling that combines rule-based rewards with LLM-as-a-judge rewards. For all tasks, we first check whether the output format conforms to DeepAnalyze’s architecture (i.e., whether it contains exactly five types of actions with the correct format). If the format is incorrect, we directly assign a reward of _R_ = _−_ 1.


key: chroma:document
string_value:
For data question answering and data-centric tasks, which have reference answers, the reward _R_ of each output _o_ are calculated using accuracy and interaction trajectory quality: 

**==> picture [189 x 24] intentionally omitted <==**

where 1 _acc_ ( _o_ ) _∈{_ 0 _,_ 1 _}_ indicates whether the result is correct, and _Sinteraction_ ( _o_ ) _∈_ [0 _,_ 1] is a score to evaluate the quality of the interaction trajectory. 

For open-ended research, the reward _R_ of each output _o_ is evaluated based on the quality of the final research report and the research process. Denoting each interaction turn in 

output _o_ as _Ti ∈ o_ , the reward _R_ is calculate as: 

**==> picture [239 x 44] intentionally omitted <==**


key: chroma:document
string_value:
output _o_ as _Ti ∈ o_ , the reward _R_ is calculate as: 

**==> picture [239 x 44] intentionally omitted <==**

where _Sreport_ ( _o_ ) is the score that evaluates the generated report from five aspects: usefulness, richness, soundness, interpretability, and readability. _|T |_ measures the interaction turns with the environment, where _N[T]_ = 10 is a hyperparameter. 1 _success_ ( _Ti_ ) indicates whether each interaction turn is successful. This reward encourages DeepAnalyze to engage in more successful interactions with the environment and to generate high-quality research report. 

Through curriculum-based agentic training, we progressively enhances DeepAnalyze’s capabilities following an easy-to-hard schedule, ultimately enabling it to autonomously accomplish a variety of data science tasks in real-world environments. 

## **3.3. Data-grounded Trajectory Synthesis**


key: chroma:document
string_value:
## **3.3. Data-grounded Trajectory Synthesis** 

The proposed curriculum-based agentic training relies on high-quality reasoning and interaction trajectory data, while such data is unfortunately scarce for data science tasks. To overcome this challange, we introduce a data-grounded trajectory synthesis framework that automatically constructs high-quality trajectory data tailored for data science tasks. The data-grounded trajectory synthesis framework consists of two parts: _Reasoning Trajectory Synthesis_ , which construct the reasoning trajectory for existing structured data instruction datasets, and _Interaction Trajectory Synthesis_ , which constructs entire data science trajectory based on structured data sources in the environment.


key: chroma:document
string_value:
**Reasoning Trajectory Synthesis.** Existing instruction datasets for structured data, such as TableQA (Li et al., 2023b; Lei et al., 2025), structured knowledge grounding (Zhuang et al., 2024), and data science code generation, is useful to improve LLM’s single capability. However, these datasets typically contain only instructions and responses, without the reasoning process. To address this limitation, 

--- end of page.page_number=6 ---

**DeepAnalyze: Agentic Large Language Models for Autonomous Data Science**


key: chroma:document
string_value:
|**Models**|**Coarse-grained Metrics**<br>**Success**<br>**Rate**<br>**Completion**<br>**Rate**|**Fine-grained Metrics**<br>**Score**<br>**VLM**<br>**F1: Data**<br>**Preparation**<br>**F2: Plot**<br>**Validity**<br>**F3: Data**<br>**Exploration**<br>**F4: Data**<br>**Visualization**<br>**F5: Data**<br>**Modeling**|
|---|---|---|
||**_Close-Source API-Based Agent_**||
|**o1-mini**<br>**GPT-4o-mini**<br>**GPT-4o**<br>**GPT-4-Turbo**<br>**Claude-3-5-Sonnet**<br>**GLM-4-Flash**|29.77<br>45.26<br>2.87<br>44.63<br>19.27<br>36.01<br>30.94<br>23.81<br>38.78<br>50.63<br>57.78<br>3.05<br>60.30<br>48.02<br>57.84<br>59.24<br>53.54<br>54.18<br>**66.31**<br>**68.44**<br>**3.91**<br>**75.93**<br>**56.14**<br>**69.33**<br>**71.35**<br>**57.67**<br>**64.51**<br>51.93<br>58.87<br>3.09<br>62.30<br>41.62<br>57.75<br>60.25<br>50.75<br>54.65<br>47.48<br>58.11<br>2.14<br>49.07<br>36.94<br>55.84<br>52.87<br>46.04<br>52.29<br>30.32<br>34.04<br>1.33<br>36.53<br>29.42<br>32.57<br>27.64<br>14.44<br>30.74||


key: chroma:document
string_value:
|**Llama-3.1-8B-Instruct**<br>**Gemma-2-9B-it**<br>**GLM-4-9B-Chat**<br>**Qwen2.5-7B-Instruct**<br>**Qwen2-7B-Instruct**<br>**Yi-1.5-9B-Chat-16K**<br>**CodeLlama-13B-Instruct**<br>**CodeLlama-7B-Instruct**<br>**StarCoder2-15B**<br>**Deepseek-Coder-6.7B-instruct**<br>**Qwen2.5-Coder-7B-Instruct**|24.73<br>33.89<br>1.29<br>38.24<br>18.25<br>21.98<br>22.89<br>25.85<br>29.69<br>7.07<br>11.00<br>1.06<br>26.16<br>16.90<br>23.81<br>18.11<br>17.15<br>12.66<br>25.72<br>30.38<br>1.69<br>31.51<br>23.15<br>28.07<br>27.19<br>19.14<br>27.57<br>43.83<br>50.74<br>1.43<br>51.18<br>36.41<br>47.25<br>45.24<br>34.77<br>45.99<br>22.84<br>25.58<br>1.16<br>30.93<br>20.78<br>28.73<br>25.87<br>7.52<br>23.52<br>38.20<br>42.35<br>0.73<br>38.14<br>36.36<br>35.64<br>37.08<br>27.79<br>38.22<br>10.49<br>14.64<br>0.04<br>11.67<br>11.34<br>9.43<br>14.43<br>5.15<br>12.64<br>2.88<br>3.97<br>0.00<br>3.53<br>2.37<br>2.57<br>1.74<br>1.59<br>3.31<br>2.07<br>2.61<br>0.07<br>2.57<br>1.81<br>1.59<br>3.43<br>1.19<br>2.33<br>37


key: chroma:document
string_value:
r>11.67<br>11.34<br>9.43<br>14.43<br>5.15<br>12.64<br>2.88<br>3.97<br>0.00<br>3.53<br>2.37<br>2.57<br>1.74<br>1.59<br>3.31<br>2.07<br>2.61<br>0.07<br>2.57<br>1.81<br>1.59<br>3.43<br>1.19<br>2.33<br>37.03<br>41.62<br>1.93<br>43.49<br>34.57<br>46.36<br>46.49<br>18.09<br>38.45<br>45.18<br>53.11<br>1.48<br>51.58<br>43.21<br>43.87<br>42.50<br>**35.23**<br>47.67||


key: chroma:document
string_value:
||**_Agentic Model_**||
|**DeepAnalyze-8B**|**59.91**<br>**66.24**<br>**2.86**<br>**71.68**<br>**67.86**<br>**58.62**<br>**69.09**<br>33.33<br>**61.11**||


key: chroma:document
string_value:
_Table 1._ Performance on DataSciBench. ‘Success Rate’ and ‘Completion Rate’ are pass rate and accuracy. ‘VLM’ and ‘F1-F5’ scores evaluate performance on various fine-grained data science sub-tasks, ‘Score’ denotes the overall performance. 

we enhance existing datasets by synthesizing complex and refined reasoning trajectories, which are used for DeepAnalyze’s single ability training.


key: chroma:document
string_value:
As shown in Figure 5(a), given the instruction–response pairs in the original dataset, the reasoning trajectory synthesis involves distillation and refinement steps. In the distillation step, we employ advanced LLMs as teacher models to extract their reasoning trajectories, whose correctness is verified by comparing the generated responses with the ground truth responses (DeepSeek-AI, 2025). To strengthen the model’s understanding of structured data, the distilled reasoning is reformulated by advanced LLMs into two complementary components: _⟨_ Analyze _⟩_ (reasoning process) and _⟨_ Understand _⟩_ (structured data understanding). Building on this, we introduce keyword-guided refinement to further enhance the reasoning trajectories with a focus on structured data. Previous works have shown that certain keywords, such as “but”/“wait”, play a crucial role in reasoning (Zhang et al., 2025a; Shen et al., 2025). Following this insight, we construct a key reasoning vocabulary and sample key


key: chroma:document
string_value:
that certain keywords, such as “but”/“wait”, play a crucial role in reasoning (Zhang et al., 2025a; Shen et al., 2025). Following this insight, we construct a key reasoning vocabulary and sample key reasoning words to insert into the reasoning trajectory, thereby improving its reasoning on structured data. Appendix B provides an example of keyword-guided refinement, where inserting keywords enhances the reasoning process by focusing more on the data, thereby improving the quality of the reasoning trajectory. Through reasoning trajectory synthesis, we can effectively leverage existing datasets to improve DeepAnalyze’s single ability in reasoning, structured data understanding, and code generation.


key: chroma:document
string_value:
**==> picture [295 x 157] intentionally omitted <==**

**==> picture [165 x 9] intentionally omitted <==**

**----- Start of picture text -----**<br>
Figure 6.  Length distribution of training data.<br>**----- End of picture text -----**<br>


**Interaction Trajectory Synthesis.** To enable DeepAnalyze to autonomously orchestrate and optimize multiple abilities in real-world environments, it is essential to construct multi-turn interaction trajectory data with the environment, yet such data is extremely scarce. In contrast, NL2SQL datasets such as Spider (Yu et al., 2018) and BIRD (Li et al., 2024) provide abundant structured data sources. To bridge this gap, we develop a multi-agent system to synthesize data science interaction trajectories from these data sources.


key: chroma:document
string_value:
The multi-agent system involves three roles: questioner, solver, and inspector. The questioner observes the data sources in the environment and accordingly formulates a data science problem, conditioned on a sampled task type (e.g., data preparation, data analysis, data modeling, data insight, or open-ended research). Simultaneously, the ques- 

--- end of page.page_number=7 ---


key: chroma:document
string_value:
--- end of page.page_number=7 ---

tioner produces a checklist that serves as the evaluation criterion, including interaction-level constraints (e.g., number of turns, code library) and environment-level constraints (e.g., whether new files are generated, detailed file name). Given the data science problem and the data sources, the solver interacts with the environment using the introduced five actions to complete the task. Finally, the inspector validates the trajectory by checking the interaction process and environmental changes against the checklist, determining whether the trajectory should be accepted. Importantly, filtering trajectories based on both interaction details and environmental changes substantially improves the quality of synthesized data. Through interaction trajectory synthesis, the high-quality multi-turn interaction data can be used for multi-ability agentic training (cold start and RL). 

## **3.4. DataScience-Instruct-500K**


key: chroma:document
string_value:
## **3.4. DataScience-Instruct-500K** 

We develop DeepAnalyze based on the constructed data in Sec.3.3. During the single-ability fine-tuning stage, we employ the reasoning trajectories built for data science, along with 100K general reasoning samples from AM-DeepSeekR1-0528-Distilled[1] . In the multi-ability agentic training stage (including both cold start and RL phases), we use the interaction trajectories constructed for data science. 

Figure 6 illustrates the length distribution of training data in both stages, with a sequence length of 8K in the first stage and 32K in the second. In terms of scale, the single-ability fine-tuning stage consists of approximately 470K samples, the cold-start phase of multiability training includes 20K samples, and the RL phase comprises 15K samples, resulting in a total of around 500K samples. We release all training data, named DataScience-Instruct-500K[2] , which can be used to train LLMs for data science tasks. 

## **4. Experiments**


key: chroma:document
string_value:
## **4. Experiments** 

## **4.1. Benchmarks** 

We conduct experiments on 12 data science benchmarks. 

**DataSciBench** (Zhang et al., 2025b) is the latest benchmark to evaluate LLM’s capabilities on the entire data science pipeline, covering data preparation, data analysis, data modeling, data visualizatoin, and data insight. 

**DSBench** (Jing et al., 2025) evaluates data analysis and modeling capabilities, comprising 540 real-world tasks collected from ModelOff and Kaggle competitions. 

> 1https://huggingface.co/datasets/ a-m-team/AM-DeepSeek-R1-0528-Distilled 

> 2https://huggingface.co/datasets/ RUC-DataLab/DataScience-Instruct-500K 

**==> picture [235 x 165] intentionally omitted <==**

_Figure 7._ Performance on DSBench (data analysis). 

**DABStep** (Egg et al., 2025) is a data agent benchmark with 450 real-world data analysis tasks designed to evaluate the multi-step reasoning abilities of agents.


key: chroma:document
string_value:
**DABStep** (Egg et al., 2025) is a data agent benchmark with 450 real-world data analysis tasks designed to evaluate the multi-step reasoning abilities of agents. 

**DABStep-Research** is a benchmark we constructed based on DABStep (Egg et al., 2025) to evaluate the capability of data science report generation. Considering that existing data science benchmarks rarely assess deep research abilities on structured data, we propose DABStep-Research to measure the capability to generate comprehensive data research reports from raw data sources. The evaluation covers five aspect: data preparation, data analysis, data insight, report generation, and open-ended data research. Please refer to Appendix A for details on its construction and cases. 

**DS-1000** (Lai et al., 2023) is a code generation benchmark containing 1000 data science problems spanning seven Python libraries such as NumPy, Pandas, Matplotlib, etc.


key: chroma:document
string_value:
**DS-1000** (Lai et al., 2023) is a code generation benchmark containing 1000 data science problems spanning seven Python libraries such as NumPy, Pandas, Matplotlib, etc. 

**TableQA Benchmarks** are a series of question-answering benchmarks based on structured tables, including WikiTQ (Pasupat & Liang, 2015), HybridQA (Chen et al., 2020), MultiHiertt (Zhao et al., 2022), OTT-QA (Chen et al., 2021a), FinQA (Chen et al., 2021b), TAT-QA (Nan et al., 2022), and HiTab (Cheng et al., 2022). 

## **4.2. Experimental Setup**


key: chroma:document
string_value:
## **4.2. Experimental Setup** 

We build DeepAnalyze-8B based on DeepSeek-R1-0528Qwen3-8B[3] as the foundation LLM. We use ms-swift (Zhao et al., 2024) and SkyRL (Liu et al., 2025) toolkit to accomplish single-ability fine-tuning and multi-ability agentic training respectively. The training data come from DataScience-Instruct-500K, as described in Sec.3.4. During inference, we employ the vLLM engine (Kwon et al., 2023) to deploy DeepAnalyze-8B for efficiency. All training and inference are conducted on NVIDIA A800 GPUs. 

> 3https://huggingface.co/deepseek-ai/ DeepSeek-R1-0528-Qwen3-8B 

--- end of page.page_number=8 ---

**DeepAnalyze: Agentic Large Language Models for Autonomous Data Science**


key: chroma:document
string_value:
|**Methods**<br>**LLM**<br>**Success (%)**<br>**Performance**<br>**Cost ($)**<br>**_Workfow-based Agent_**<br>**AutoGen**<br>Llama3-8b<br>5.41<br>1.55<br>0.00<br>Llama3-70b<br>16.22<br>7.79<br>0.00<br>GPT-3.5<br>8.11<br>6.02<br>0.41<br>GPT-4<br>87.84<br>**45.52**<br>19.34<br>GPT-4o<br>71.62<br>34.74<br>12.27<br>GPT-4o-mini<br>22.97<br>11.24<br>0.10<br>**Code**<br>**Interpreter**<br>GPT-3.5<br>16.22<br>6.52<br>2.74<br>GPT-4<br>54.05<br>26.14<br>38.81<br>GPT-4o<br>44.59<br>19.87<br>19.26<br>GPT-4o-mini<br>39.19<br>16.90<br>2.70<br>**_Agentic Model_**<br>**DeepAnalyze-8B**<br>**90.63**<br>39.41<br>**0.00**<br>_Table 2._ Performance on DSBench (data modeling).<br>**4.3. Main Results**<br>**Capability on End-to-end Data Science Pipeline.**<br>We|**Methods**<br>**LLM**<br>**Easy Level**<br>**(72 Cases)**<br>**Hard Level**<br>**(378 Cases)**<br>**Overall**<br>**(450 Cases)**|
|---|---|
||**_Workfow-based Agent_**|


key: chroma:document
string_value:
|---|---|
||**_Workfow-based Agent_**|
||**ReAct**<br>Llama-4-Scout<br>52.78<br>1.85<br>10.00<br>Qwen3-Coder<br>54.17<br>3.44<br>11.56<br>GPT-4o-mini<br>69.44<br>3.44<br>14.00<br>Deepseek-v3<br>66.67<br>5.56<br>15.34<br>GPT-4o<br>66.67<br>6.08<br>15.77<br>Claude-3.5-Haiku<br>77.78<br>5.03<br>16.67<br>Llama-4-Maverick<br>75.00<br>8.73<br>19.33<br>GPT-4.1-mini<br>77.78<br>8.99<br>20.00<br>Claude-3.5-Sonnet<br>77.78<br>9.26<br>20.22<br>GPT-4.1<br>80.56<br>12.43<br>23.33|
||**Reasoning**<br>**Prompt**<br>o1<br>69.44<br>11.11<br>20.44<br>Gemini-2.5-Pro<br>66.67<br>12.70<br>21.34<br>o3-mini<br>72.22<br>13.76<br>23.11<br>o4-mini<br>76.39<br>14.55<br>24.44|
||**DS-Agent**<br>Gemini-2.0-Flash<br>61.11<br>9.79<br>18.00<br>**Open Data Scientist**<br>Deepseek-v3<br>**84.72**<br>16.40<br>27.33<br>**I2I-Agent**<br>Claude-3.5-Sonnet<br>80.56<br>28.04<br>36.44|
||**_Agentic Model_**|
||**DeepAnalyze-8B**<br>70.83<br>**32.80**<br>**38.88**|


key: chroma:document
string_value:
**Capability on End-to-end Data Science Pipeline.** We evaluate DeepAnalyze on DataSciBench to assess its _end-toend data science capabilities_ , where each problem involves one or more sub-tasks such as data preparation, analysis, modeling, and visualization. We compare DeepAnalyze-8B with several workflow-based (ReAct) agents, covering 17 open-source and advanced proprietary LLMs. As shown in Table 1, coarse-grained metrics measure task success and sub-task completion rates, while fine-grained metrics evaluate detailed performance across individual stages of the data science pipeline. The results show that, despite having only 8B parameters, DeepAnalyze-8B achieves state-of-the-art performance among open-source LLM-based agents and even outperforms most advanced proprietary models (e.g., GPT-4-Turbo, GPT-4o-mini, Claude 3.5 Sonnet), ranking second only to GPT-4o. More importantly, unlike existing workflow-based agents, DeepAnalyze-8B accomplishes high-quality, end-to-end pipelines


key: chroma:document
string_value:
GPT-4-Turbo, GPT-4o-mini, Claude 3.5 Sonnet), ranking second only to GPT-4o. More importantly, unlike existing workflow-based agents, DeepAnalyze-8B accomplishes high-quality, end-to-end pipelines without relying on external orchestration frameworks such as ReAct. Prior studies have shown that models like o1-mini exhibit strong reasoning ability but often fail to execute complex data science tasks requiring precise instruction following and strategic planning (Zhang et al., 2025b). In contrast, DeepAnalyze benefits from agentic training, enabling autonomous orchestration and adaptive optimization in real-world environments, resulting in consistently superior performance.


key: chroma:document
string_value:
_Table 3._ Performance on DABStep benchmark. 

adaptive optimization are more effective than the manually designed workflows used in agents such as Code Interpreter, Master-Slave (Kong et al., 2017), and Blackboard (Salemi et al., 2025). We then evaluate its _data modeling_ capabilities. Table 2 reports the results on DSBench, where tasks involve training machine learning models (Jing et al., 2025). DeepAnalyze-8B achieves performance comparable to AutoGen-based workflows (Wu et al., 2024) built upon various advanced proprietary LLMs. Although it has fewer parameters and weaker single-turn reasoning ability, DeepAnalyze-8B can autonomously optimize its actions through environment feedback, achieving a high task success rate and strong overall performance.


key: chroma:document
string_value:
To further evaluate DeepAnalyze’s ability to perform data analysis across multiple data types, including structured, semi-structured, and unstructured data (Egg et al., 2025), we evaluate it on DABStep, which contains diverse data formats such as markdown, CSV, and JSON. As shown in Table 3, DeepAnalyze-8B outperforms previous workflowbased agents, including ReAct (Yao et al., 2023), reasoning prompts, and specially designed workflows, particularly on hard-level tasks. While workflow-based systems can leverage the strong general capabilities of proprietary LLMs to perform well on easy tasks, their predefined workflows limit performance on complex scenarios. In contrast, DeepAnalyze, equipped with autonomous orchestration and adaptive optimization through agentic training, can iteratively interact with the environment like a human data scientist, achieving superior performance on complex tasks requiring long-chain reasoning.


key: chroma:document
string_value:
Overall, DeepAnalyze-8B’s strong results on DataSciBench highlight its advanced problem-solving capabilities in autonomously orchestrating end-to-end data science pipelines. 

**Capability on Individual Data Science Tasks.** As most previous studies primarily focus on individual data science tasks such as data analysis and modeling, we further evaluate DeepAnalyze on these tasks using DSBench for a fair comparison. We first evaluate its _statistical data analysis_ capabilities. As shown in Figure 7, DeepAnalyze-8B outperforms previous LLM prompting and workflow-based agents, demonstrating that its autonomous orchestration and 

**Capability on Data-Oriented Deep Research.** Deep research has emerged as an important task for evaluating the comprehensive capabilities of LLMs and agents. To this end, 

--- end of page.page_number=9 ---

**==> picture [487 x 126] intentionally omitted <==**


key: chroma:document
string_value:
--- end of page.page_number=9 ---

**==> picture [487 x 126] intentionally omitted <==**

**----- Start of picture text -----**<br>
Figure 8.  Performance on DABStep-Research.<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
|**Models**|**WikiTQ**|**HybridQA**|**MultiHiertt**|**OTT-QA**|**FinQA**|**TAT-QA**|**HiTab**|**AVG**|
|---|---|---|---|---|---|---|---|---|
|||**API-based LLMs**|||||||
|**Claude**|82.02|39.36|40.98|62.69|57.45|53.09|75.96|58.79|
|**GPT-4o**|81.19|39.30|40.86|66.35|57.63|53.45|73.92|58.96|
|||**Open-Source LLMs**|||||||
|**DeepSeek-R1-0528**|**84.00**|39.04|40.98|66.85|59.90|55.24|75.57|60.22|
|**TableGPT2-7B**|63.70|30.03|25.12|48.87|38.36|55.12|63.89|46.44|
|**Qwen2.5-32B-Inst**|79.65|38.20|37.74|56.50|59.20|67.29|73.29|58.84|
|**Qwen2.5-7B-Inst**|57.27|31.84|27.54|50.50|52.40|49.79|57.19|46.65|
|**DeepSeek-R1-0528-Qwen3-8B**|63.49|28.15|39.86|49.72|51.09|55.00|51.09|48.34|
|**Reasoning-Table (SFT)**|72.35|35.17|38.50|54.40|60.42|63.45|72.72|56.72|
|**Reasoning-Table (SFT+RL)**|75.46|42.83|39.56|**68.68**|**64.46**|**73.75**|73.61|62.62|
|**DeepAnalyze-8B (single-ability)**|81.86|39.27|44.58|53.12|62.50|66.87|76.26|60.64|


key: chroma:document
string_value:
|**Reasoning-Table (SFT+RL)**|75.46|42.83|39.56|**68.68**|**64.46**|**73.75**|73.61|62.62|
|**DeepAnalyze-8B (single-ability)**|81.86|39.27|44.58|53.12|62.50|66.87|76.26|60.64|
|**DeepAnalyze-8B**|83.24|**42.95**|**48.29**|64.73|63.30|70.64|**78.16**|**64.47**|


key: chroma:document
string_value:
_Table 4._ Performance on TableQA benchmarks. ‘DeepAnalyze-8B (single-ability)’ is the model after the first stage fine-tuning. 

we introduce DABStep-Research, a benchmark designed to evaluate the _data-oriented deep research_ capabilities of LLMs and agents. We compare DeepAnalyze-8B with advanced agent systems (i.e., state-of-the-art proprietary LLMs with tool-calling capabilities) on a suite of data research tasks spanning five categories: data preparation, data analysis, data insight, report generation (with a specified outline), and open-ended data research (fully unconstrained). Each task results in a research report, which is evaluated on both content quality and formatting. Figure 9 illustrates several representative cases from DABStep-Research.


key: chroma:document
string_value:
The results in Figure 8 show that DeepAnalyze-8B consistently outperforms all compared systems across every task. Notably, agent systems built on proprietary LLMs with tool calls exhibit a significant performance drop on open-ended data research tasks compared to more instructive tasks, such as data preparation, analysis, and insight, where explicit steps or goals are provided. This decline stems from their lack of training in data science: without step-by-step guidance, they fail to perform autonomous orchestration and adaptive optimization. In contrast, DeepAnalyze-8B, trained in real-world environments, effectively handles fully openended data research tasks without predefined instructions. Moreover, it achieves a clear advantage in report format quality, generating outputs that closely resemble analystgrade reports. This improvement is attributed to reward


key: chroma:document
string_value:
modeling that explicitly incorporates report quality during RL training. Appendix C further provides qualitative comparisons of research reports generated by DeepAnalyze-8B and reasoning models such as DeepSeek-R1 and o3-mini, highlighting DeepAnalyze-8B’s superior content depth and structured presentation. 

Overall, DeepAnalyze-8B enables end-to-end autonomous data research, from raw data to analyst-grade reports, unlocking novel applications in data research.


key: chroma:document
string_value:
Overall, DeepAnalyze-8B enables end-to-end autonomous data research, from raw data to analyst-grade reports, unlocking novel applications in data research. 

**Capability Related to Data Science.** Beside data science tasks, we further evaluate DeepAnalyze-8B on DS-1000 and TableQA to evaluate its capabilities in code generation and structured data understanding, which are essential for complex data science. As reported in Table 5 and Table 4, DeepAnalyze-8B outperforms GPT-4-Turbo and GLM-4.5 (GLM-4.5-Team, 2025) on DS-1000, and surpasses the previous SOTA model Reasoning-Table (Lei et al., 2025) on TableQA. Compared with DeepSeek-R1-0528-Qwen3-8B, DeepAnalyze-8B achieves substantial gains in both abilities under the single-ability setting, demonstrating the effectiveness of the first-stage single-ability fine-tuning. Furthermore, agentic training on complex data science tasks further strengthens these specialized capabilities.


key: chroma:document
string_value:
Overall, DeepAnalyze-8B’s strong performance on code generation and structured data understanding establishes a 

--- end of page.page_number=10 ---

**DeepAnalyze: Agentic Large Language Models for Autonomous Data Science**


key: chroma:document
string_value:
|**Models**|**Data Science Libraries**<br>**Overall**<br>**Pandas**<br>**NumPy**<br>**Matplotlib**<br>**Scikit-learn**<br>**SciPy**<br>**TensorFlow**<br>**PyTorch**|
|---|---|
|**Codex002**<br>**GPT-3.5-turbo**<br>**GPT-4**<br>**GPT-4-turbo**<br>**Kimi-K2-Instruct**_∗_<br>**GLM-4.5**_∗_<br>**LIMI**_∗_<br>**DeepSeek-R1-0528-Qwen3-8B**|26.5<br>43.2<br>54.8<br>43.5<br>34.9<br>37.8<br>39.7<br>38.8<br>33.0<br>36.8<br>58.7<br>35.7<br>39.6<br>33.3<br>29.4<br>38.6<br>41.9<br>56.8<br>65.2<br>50.4<br>48.1<br>46.7<br>47.1<br>51.0<br>42.3<br>61.8<br>**71.6**<br>50.4<br>50.0<br>53.3<br>50.0<br>53.9<br>-<br>-<br>-<br>-<br>-<br>-<br>-<br>40.2<br>-<br>-<br>-<br>-<br>-<br>-<br>-<br>53.2<br>-<br>-<br>-<br>-<br>-<br>-<br>-<br>54.8<br>17.5<br>37.3<br>52.9<br>27.8<br>21.7<br>31.1<br>29.4<br>30.4|


key: chroma:document
string_value:
|**DeepAnalyze-8B (single-ability)**<br>**DeepAnalyze-8B**|43.6<br>69.1<br>54.8<br>53.0<br>50.9<br>64.4<br>58.8<br>54.8<br>**50.2**<br>**74.5**<br>67.7<br>**56.5**<br>**54.7**<br>**68.9**<br>**70.6**<br>**61.7**|


key: chroma:document
string_value:
_Table 5._ Performance on DS-1000. _[∗]_ indicates that The results are derived from corresponding references. ‘DeepAnalyze-8B (singleability)’ is the model after the first stage fine-tuning. 

|**Models**|**WikiTQ**|**MultiHiertt**|**DS-1000**|**DABStep**|
|---|---|---|---|---|
|**DeepAnalyze**|83.24|48.29|61.70|38.88|
|**- w/o**_ ⟨_**Understand**_⟩_|80.78|45.43|61.20|31.78|



_Table 6._ Ablation study on _⟨_ Understand _⟩_ action. 

|**Training Methods**<br>**Curriculum-based Agentic Training**<br>**-Only Single-ability Fine-tuning**|**WikiTQ **<br>**83.24**<br>81.86|** MultiHiertt **<br>**48.29**<br>44.58|** DS-1000 **<br>**61.70**<br>54.80|** DABStep**<br>**38.88**<br>15.34|
|---|---|---|---|---|
|**-Only Multi-ability Agentic Training**|80.32|43.29|53.20|30.66|
|**-One-stage Training**|82.13|46.23|54.80|36.89|



_Table 7._ Ablation study on the curriculum-based agentic training. 

robust foundation for its advanced performance in end-toend autonomous data science.


key: chroma:document
string_value:
_Table 7._ Ablation study on the curriculum-based agentic training. 

robust foundation for its advanced performance in end-toend autonomous data science. 

## **5. Analysis** 

## **5.1. Ablation on DeepAnalyze’s Actions** 

DeepAnalyze introduces five actions for autonomous data science, among which _⟨_ Understand _⟩_ is specifically designed for structured data understanding. To evaluate the effect of incorporating _⟨_ Understand _⟩_ independently from reasoning process (i.e., _⟨_ Analyze _⟩_ ), we conduct an ablation study, as reported in Table 6. The results show that removing _⟨_ Understand _⟩_ leads to performance drops on structured data understanding tasks (WikiTQ, MultiHiertt) as well as data analysis tasks (DABStep), demonstrating the advantage of introducing _⟨_ Understand _⟩_ in DeepAnalyze. 

## **5.2. Superiority of Curriculum-based Agentic Training**


key: chroma:document
string_value:
## **5.2. Superiority of Curriculum-based Agentic Training** 

To address the challenges arising from the multiple ability requirements in data science, we introduce curriculum-based agentic training, inspired by the learning path of human data scientists, where first fine-tuning on single abilities and then agentic training on complex tasks that require multiple abilities. To evaluate its effectiveness, we compare several training methods, including “Only Single-ability Fine-tuning”, “Only Multi-ability Agentic Training”, and “One-stage Training”, which directly mix the single-ability 

|**Reasoning Trajectory**|**WikiTQ**|**HybridQA**|** MultiHiertt**|**HiTab**|
|---|---|---|---|---|
|**Original**|75.54|34.42|39.29|72.95|
|**+ Distillation**|78.80|36.12|41.24|74.44|
|**+ Distillation + Refnement**|**80.25**|**38.84**|**43.47**|**75.86**|



_Table 8._ Performance under various reasoning trajectory synthesis.


key: chroma:document
string_value:
_Table 8._ Performance under various reasoning trajectory synthesis. 

data into the cold-start of multi-ability agentic training (i.e., the conventional agentic training methods). 

As shown in Table 7, “Only Single-ability Fine-tuning” fails to handle complex tasks in DABStep that require multi-turn interaction with the environment, and “Only Multi-ability Agentic Training” struggles to achieve strong performance when single ability are not well established. Compared with “One-stage Training”, a scheduled training process from simple (single-ability) to complex (multi-ability) proves more beneficial for model performance using the same data. Therefore, for tasks that rely on multiple abilities, curriculum-based agentic training effectively enhances overall model performance. 

## **5.3. Advantage of Reasoning Trajectory Synthesis**


key: chroma:document
string_value:
## **5.3. Advantage of Reasoning Trajectory Synthesis** 

During data synthesis, we propose reasoning trajectory synthesis that incorporates distillation and refinement to enhance the model’s reasoning ability over structured data. To validate its effectiveness, we compare the model’s performance when trained on original, distilled, and refined data, where the original data are derived from Reasoning-Table. As reported in Table 1, both distillation and refinement improve the model’s understanding of structured data. In particular, compared with commonly used distillation methods, we additionally introduce a refinement stage, which incorporates key reasoning vocabulary to strengthen the reasoning trajectory’s focus on structured data, thereby improving the overall data quality. 

## **6. Conclusion and Future Work** 

DeepAnalyze brings a major leap forward in autonomous data science, demonstrating unprecedented capabilities 

--- end of page.page_number=11 ---


key: chroma:document
string_value:
## **6. Conclusion and Future Work** 

DeepAnalyze brings a major leap forward in autonomous data science, demonstrating unprecedented capabilities 

--- end of page.page_number=11 ---

across a wide spectrum of data-centric tasks. Powered by curriculum-based agentic training and data-grounded trajectory synthesis, _DeepAnalyze-8B outperforms state-ofthe-art closed-source LLMs on 12 data science benchmarks_ . 

More importantly, DeepAnalyze goes beyond predefined workflows, as it enables open-ended data research and generates analyst-grade reports, advancing a long-standing goal of the data science community: automatically extracting actionable insights from raw data. As a result, this work marks _a paradigm shift_ in autonomous data science from workflow-based agents to agentic models, paving the way for the next generation of intelligent data systems in areas such as data discovery, data governance, data ecosystems, and data management. 

## **References**


key: chroma:document
string_value:
## **References** 

- Chen, W., Zha, H., Chen, Z., Xiong, W., Wang, H., and Wang, W. Y. HybridQA: A dataset of multi-hop question answering over tabular and textual data. In Cohn, T., He, Y., and Liu, Y. (eds.), _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pp. 1026–1036, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020. findings-emnlp.91. URL https://aclanthology. org/2020.findings-emnlp.91/. 

- Chen, W., Chang, M.-W., Schlinger, E., Wang, W. Y., and Cohen, W. W. Open question answering over tables and text. In _International Conference on Learning Representations_ , 2021a. URL https://openreview.net/ forum?id=MmCRswl1UYl.


key: chroma:document
string_value:
- Chen, Z., Chen, W., Smiley, C., Shah, S., Borova, I., Langdon, D., Moussa, R., Beane, M., Huang, T.H., Routledge, B., and Wang, W. Y. FinQA: A dataset of numerical reasoning over financial data. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.t. (eds.), _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pp. 3697–3711, Online and Punta Cana, Dominican Republic, November 2021b. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main. 300. URL https://aclanthology.org/2021. emnlp-main.300/.


key: chroma:document
string_value:
- Cheng, Z., Dong, H., Wang, Z., Jia, R., Guo, J., Gao, Y., Han, S., Lou, J.-G., and Zhang, D. HiTab: A hierarchical table dataset for question answering and natural language generation. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 1094–1110, Dublin, Ireland, May 2022. Association for Computational Linguis- 

- tics. doi: 10.18653/v1/2022.acl-long.78. URL https: //aclanthology.org/2022.acl-long.78/. 

- De Bie, T., De Raedt, L., Hernández-Orallo, J., Hoos, H. H., Smyth, P., and Williams, C. K. Automating data science. _Communications of the ACM_ , 65(3):76–87, 2022. 

- DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948.


key: chroma:document
string_value:
- DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. 

- Egg, A., Goyanes, M. I., Kingma, F., Mora, A., von Werra, L., and Wolf, T. Dabstep: Data agent benchmark for multi-step reasoning, 2025. URL https://arxiv. org/abs/2506.23719. 

- Fang, X., Xu, W., Tan, F. A., Hu, Z., Zhang, J., Qi, Y., Sengamedu, S. H., and Faloutsos, C. Large language models (LLMs) on tabular data: Prediction, generation, and understanding - a survey. _Transactions on Machine Learning Research_ , 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=IZnrCGF9WI. 

- GLM-4.5-Team. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models, 2025. URL https://arxiv. org/abs/2508.06471. 

- Guo, S., Deng, C., Wen, Y., Chen, H., Chang, Y., and Wang, J. Ds-agent: Automated data science by empowering large language models with case-based reasoning. In _ICML_ , 2024. URL https://openreview.net/ forum?id=LfJgeBNCFI.


key: chroma:document
string_value:
- Hollmann, N., Müller, S., and Hutter, F. Large language models for automated data science: Introducing caafe for context-aware automated feature engineering. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), _Advances in Neural Information Processing Systems_ , volume 36, pp. 44753–44775. Curran Associates, Inc., 2023. 

- Hong, S., Lin, Y., Liu, B., Liu, B., Wu, B., Zhang, C., Wei, C., Li, D., Chen, J., Zhang, J., Wang, J., Zhang, L., Zhang, L., Yang, M., Zhuge, M., Guo, T., Zhou, T., Tao, W., Tang, X., Lu, X., Zheng, X., Liang, X., Fei, Y., Cheng, Y., Gou, Z., Xu, Z., and Wu, C. Data interpreter: An llm agent for data science, 2024. URL https://arxiv.org/ abs/2402.18679.


key: chroma:document
string_value:
- Hong, S., Lin, Y., Liu, B., Liu, B., Wu, B., Zhang, C., Li, D., Chen, J., Zhang, J., Wang, J., Zhang, L., Zhang, L., Yang, M., Zhuge, M., Guo, T., Zhou, T., Tao, W., Tang, R., Lu, X., Zheng, X., Liang, X., Fei, Y., Cheng, Y., Ni, Y., Gou, Z., Xu, Z., Luo, Y., and Wu, C. Data interpreter: An LLM agent for data science. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), _Findings of the Association for Computational Linguistics: ACL 2025_ , pp. 19796–19821, Vienna, Austria, July 

--- end of page.page_number=12 ---

2025. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025.findings-acl. 1016. URL https://aclanthology.org/2025. findings-acl.1016/.


key: chroma:document
string_value:
2025. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025.findings-acl. 1016. URL https://aclanthology.org/2025. findings-acl.1016/. 

- Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, X., and Wen, J.-R. StructGPT: A general framework for large language model to reason over structured data. In Bouamor, H., Pino, J., and Bali, K. (eds.), _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pp. 9237–9251, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main. 574. URL https://aclanthology.org/2023. emnlp-main.574/. 

- Jin, B., Zeng, H., Yue, Z., Yoon, J., Arik, S., Wang, D., Zamani, H., and Han, J. Search-r1: Training llms to reason and leverage search engines with reinforcement learning, 2025. URL https://arxiv.org/abs/ 2503.09516.


key: chroma:document
string_value:
- Jing, L., Huang, Z., Wang, X., Yao, W., Yu, W., Ma, K., Zhang, H., Du, X., and Yu, D. DSBench: How far are data science agents from becoming data science experts? In _The Thirteenth International Conference on Learning Representations_ , 2025. URL https://openreview. net/forum?id=DSsSPr0RZJ. 

- Kong, X., Xin, B., Liu, F., and Wang, Y. Revisiting the master-slave architecture in multi-agent deep reinforcement learning, 2017. URL https://arxiv.org/ abs/1712.07305. 

- Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In _Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles_ , 2023.


key: chroma:document
string_value:
- Lai, Y., Li, C., Wang, Y., Zhang, T., Zhong, R., Zettlemoyer, L., Yih, W.-T., Fried, D., Wang, S., and Yu, T. DS-1000: A natural and reliable benchmark for data science code generation. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), _Proceedings of the 40th International Conference on Machine Learning_ , volume 202 of _Proceedings of Machine Learning Research_ , pp. 18319–18345. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/ v202/lai23b.html. 

- Lei, F., Meng, J., Huang, Y., Chen, T., Zhang, Y., He, S., Zhao, J., and Liu, K. Reasoning-table: Exploring reinforcement learning for table reasoning, 2025. URL https://arxiv.org/abs/2506.01710. 

- Li, J., Hui, B., Qu, G., Yang, J., Li, B., Li, B., Wang, B., Qin, B., Geng, R., Huo, N., et al. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. _Advances in Neural Information Processing Systems_ , 36, 2024.


key: chroma:document
string_value:
- Li, P., He, Y., Yan, C., Wang, Y., and Chaudhuri, S. Autotables: Synthesizing multi-step transformations to relationalize tables without using examples, 2023a. URL https://arxiv.org/abs/2307.14565. 

- Li, P., He, Y., Yashar, D., Cui, W., Ge, S., Zhang, H., Fainman, D. R., Zhang, D., and Chaudhuri, S. Table-gpt: Table-tuned gpt for diverse table tasks, 2023b. URL https://arxiv.org/abs/2310.09263. 

- Li, X., Dong, G., Jin, J., Zhang, Y., Zhou, Y., Zhu, Y., Zhang, P., and Dou, Z. Search-o1: Agentic search-enhanced large reasoning models, 2025. URL https://arxiv. org/abs/2501.05366. 

- Liu, S., Hegde, S., Cao, S., Zhu, A., Li, D., Griggs, T., Tang, E., Malik, A., Hakhamaneshi, K., Liaw, R., Moritz, P., Zaharia, M., Gonzalez, J. E., and Stoica, I. Skyrl-sql: Matching gpt-4o and o4-mini on text2sql with multi-turn rl, 2025.


key: chroma:document
string_value:
- Liu, X., Shen, S., Li, B., Ma, P., Jiang, R., Zhang, Y., Fan, J., Li, G., Tang, N., and Luo, Y. A survey of nl2sql with large language models: Where are we, and where are we going? _arXiv preprint arXiv:2408.05109_ , 2024. 

- Mohammadjafari, A., Maida, A. S., and Gottumukkala, R. From natural language to sql: Review of llm-based textto-sql systems, 2025. URL https://arxiv.org/ abs/2410.01066. 

- Nan, L., Hsieh, C., Mao, Z., Lin, X. V., Verma, N., Zhang, R., Kry´sci´nski, W., Schoelkopf, H., Kong, R., Tang, X., Mutuma, M., Rosand, B., Trindade, I., Bandaru, R., Cunningham, J., Xiong, C., Radev, D., and Radev, D. FeTaQA: Free-form table question answering. _Transactions of the Association for Computational Linguistics_ , 10:35– 49, 2022. doi: 10.1162/tacl_a_00446. URL https: //aclanthology.org/2022.tacl-1.3/.


key: chroma:document
string_value:
- Nascimento, N., Guimaraes, E., Chintakunta, S. S., and Boominathan, S. A. Llm4ds: Evaluating large language models for data science code generation, 2024. URL https://arxiv.org/abs/2411.11908. 

- Nejjar, M., Zacharias, L., Stiehle, F., and Weber, I. Llms for science: Usage for code generation and data analysis, 2024. URL https://arxiv.org/abs/2311. 16733. 

OpenAI. Gpt-4 technical report, 2023. 

--- end of page.page_number=13 ---

- OpenAI. Openai o1 system card, 2024. URL https: //arxiv.org/abs/2412.16720.


key: chroma:document
string_value:
OpenAI. Gpt-4 technical report, 2023. 

--- end of page.page_number=13 ---

- OpenAI. Openai o1 system card, 2024. URL https: //arxiv.org/abs/2412.16720. 

- Ouyang, G., Chen, J., Nie, Z., Gui, Y., Wan, Y., Zhang, H., and Chen, D. nvAgent: Automated data visualization from natural language via collaborative agent workflow. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), _Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 19534–19567, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long. 960. URL https://aclanthology.org/2025. acl-long.960/.


key: chroma:document
string_value:
- Pan, B., Fu, Y., Wang, K., Lu, J., Pan, L., Qian, Z., Chen, Y., Wang, G., Zhou, Y., Zheng, L., Tang, Y., Wen, Z., Wu, Y., Lu, J., Zhu, B., Zhu, M., Zhang, B., and Chen, W. Vis-shepherd: Constructing critic for llm-based data visualization generation, 2025. URL https://arxiv. org/abs/2506.13326. 

- Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., and Wang, W. Y. Automatically correcting large language models: Surveying the landscape of diverse selfcorrection strategies, 2023. URL https://arxiv. org/abs/2308.03188. 

- Pasupat, P. and Liang, P. Compositional semantic parsing on semi-structured tables. In Zong, C. and Strube, M. (eds.), _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pp. 1470–1480, Beijing, China, July 2015. Association for Computational Linguistics. doi: 10.3115/v1/P15-1142. URL https://aclanthology.org/P15-1142/.


key: chroma:document
string_value:
- Plaat, A., van Duijn, M., van Stein, N., Preuss, M., van der Putten, P., and Batenburg, K. J. Agentic large language models, a survey, 2025. URL https://arxiv.org/ abs/2503.23037. 

- Ren, Z. Z., Shao, Z., Song, J., Xin, H., Wang, H., Zhao, W., Zhang, L., Fu, Z., Zhu, Q., Yang, D., Wu, Z. F., Gou, Z., Ma, S., Tang, H., Liu, Y., Gao, W., Guo, D., and Ruan, C. Deepseek-prover-v2: Advancing formal mathematical reasoning via reinforcement learning for subgoal decomposition, 2025. URL https://arxiv. org/abs/2504.21801. 

- Salemi, A., Parmar, M., Goyal, P., Song, Y., Yoon, J., Zamani, H., Palangi, H., and Pfister, T. Llm-based multiagent blackboard system for information discovery in data science, 2025. URL https://arxiv.org/abs/ 2510.01285.


key: chroma:document
string_value:
- Sapkota, R., Roumeliotis, K. I., and Karkee, M. Ai agents vs. agentic ai: A conceptual taxonomy, applications and challenges. _Information Fusion_ , 126:103599, February 2025a. ISSN 1566-2535. doi: 10.1016/j.inffus.2025. 103599. URL http://dx.doi.org/10.1016/j. inffus.2025.103599. 

- Sapkota, R., Roumeliotis, K. I., and Karkee, M. Vibe coding vs. agentic coding: Fundamentals and practical implications of agentic ai, 2025b. URL https: //arxiv.org/abs/2505.19443. 

- Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300. 

- Shen, S., Huang, F., Zhao, Z., Liu, C., Zheng, T., and Zhu, D. Long is more important than difficult for training reasoning models, 2025. URL https://arxiv.org/ abs/2503.18069.


key: chroma:document
string_value:
- Shen, S., Huang, F., Zhao, Z., Liu, C., Zheng, T., and Zhu, D. Long is more important than difficult for training reasoning models, 2025. URL https://arxiv.org/ abs/2503.18069. 

- Sun, M., Han, R., Jiang, B., Qi, H., Sun, D., Yuan, Y., and and, J. H. Lambda: A large model based data agent. _Journal of the American Statistical Association_ , 0(ja):1–20, 2025a. doi: 10.1080/01621459.2025. 2510000. URL https://doi.org/10.1080/ 01621459.2025.2510000. 

- Sun, M., Han, R., Jiang, B., Qi, H., Sun, D., Yuan, Y., and Huang, J. A survey on large language model-based agents for statistics and data science, 2025b. URL https: //arxiv.org/abs/2412.14222. 

- Wang, P., Yu, Y., Chen, K., Zhan, X., and Wang, H. Large language model-based data science agent: A survey, 2025. URL https://arxiv.org/abs/2508.02744.


key: chroma:document
string_value:
- Wang, P., Yu, Y., Chen, K., Zhan, X., and Wang, H. Large language model-based data science agent: A survey, 2025. URL https://arxiv.org/abs/2508.02744. 

- Wen, Y., Yin, P., Shi, K., Michalewski, H., Chaudhuri, S., and Polozov, A. Grounding data science code generation with input-output specifications, 2024. URL https: //arxiv.org/abs/2402.08073. 

- Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H., White, R. W., Burger, D., and Wang, C. Autogen: Enabling next-gen LLM applications via multi-agent conversations. In _First Conference on Language Modeling_ , 2024. URL https://openreview.net/forum? id=BAakY1hNKS. 

- Xu, Y., He, S., Chen, J., Xiangrong, Z., Wang, B., Liu, G., Zhao, J., and Liu, K. Llasa: Large language and structured data assistant, 2025. URL https://arxiv. org/abs/2411.14460. 

--- end of page.page_number=14 ---


key: chroma:document
string_value:
--- end of page.page_number=14 ---

- Xue, S., Jiang, C., Shi, W., Cheng, F., Chen, K., Yang, H., Zhang, Z., He, J., Zhang, H., Wei, G., Zhao, W., Zhou, F., Qi, D., Yi, H., Liu, S., and Chen, F. Db-gpt: Empowering database interactions with private large language models, 2024. URL https://arxiv.org/abs/2312. 17449. 

- Yang, J., He, Y., and Chaudhuri, S. Auto-pipeline: Synthesizing complex data pipelines by-target using reinforcement learning and search. volume 14, pp. 2563 – 2576, 2021. 

- Yang, Z., Zhou, Z., Wang, S., Cong, X., Han, X., Yan, Y., Liu, Z., Tan, Z., Liu, P., Yu, D., Liu, Z., Shi, X., and Sun, M. MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization. In _Findings of the Association for Computational Linguistics ACL 2024_ , pp. 11789–11804, Stroudsburg, PA, USA, aug 2024. Association for Computational Linguistics. ISBN 9798400704901. doi: 10.18653/v1/2024.findings-acl. 701.


key: chroma:document
string_value:
- Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. React: Synergizing reasoning and acting in language models. In _The Eleventh International Conference on Learning Representations_ , 2023. URL https://openreview.net/forum? id=WE_vluYUL-X. 

- Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., Zhang, Z., and Radev, D. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (eds.), _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 3911–3921, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1425. URL https://aclanthology.org/D18-1425/.


key: chroma:document
string_value:
- Zhang, A., Chen, Y., Pan, J., Zhao, C., Panda, A., Li, J., and He, H. Reasoning models know when they’re right: Probing hidden states for self-verification, 2025a. URL https://arxiv.org/abs/2504.05419. 

- Zhang, D., Wu, J., Lei, J., Che, T., Li, J., Xie, T., Huang, X., Zhang, S., Pavone, M., Li, Y., Ouyang, W., and Zhou, D. Llama-berry: Pairwise optimization for o1-like olympiadlevel mathematical reasoning, 2024. URL https:// arxiv.org/abs/2410.02884. 

- Zhang, D., Zhoubian, S., Cai, M., Li, F., Yang, L., Wang, W., Dong, T., Hu, Z., Tang, J., and Yue, Y. Datascibench: An llm agent benchmark for data science, 2025b. URL https://arxiv.org/abs/2502.13897.


key: chroma:document
string_value:
- Zhang, X., Luo, S., Zhang, B., Ma, Z., Zhang, J., Li, Y., Li, G., Yao, Z., Xu, K., Zhou, J., Zhang-Li, D., Yu, J., Zhao, S., Li, J., and Tang, J. TableLLM: Enabling tabular data manipulation by LLMs in real office usage scenarios. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.), _Findings of the Association for Computational Linguistics: ACL 2025_ , pp. 10315–10344, Vienna, Austria, July 2025c. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025. findings-acl.538. URL https://aclanthology. org/2025.findings-acl.538/.


key: chroma:document
string_value:
- Zhao, Y., Li, Y., Li, C., and Zhang, R. MultiHiertt: Numerical reasoning over multi hierarchical tabular and textual data. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 6588–6600, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.454. URL https: //aclanthology.org/2022.acl-long.454/. 

- Zhao, Y., Huang, J., Hu, J., Wang, X., Mao, Y., Zhang, D., Jiang, Z., Wu, Z., Ai, B., Wang, A., Zhou, W., and Chen, Y. Swift:a scalable lightweight infrastructure for fine-tuning, 2024. URL https://arxiv.org/abs/ 2408.05517. 

- Zheng, Y., Fu, D., Hu, X., Cai, X., Ye, L., Lu, P., and Liu, P. Deepresearcher: Scaling deep research via reinforcement learning in real-world environments, 2025. URL https: //arxiv.org/abs/2504.03160.


key: chroma:document
string_value:
- Zhuang, A., Zhang, G., Zheng, T., Du, X., Wang, J., Ren, W., Huang, S. W., Fu, J., Yue, X., and Chen, W. Structlm: Towards building generalist models for structured knowledge grounding, 2024. URL https: //arxiv.org/abs/2402.16671. 

--- end of page.page_number=15 ---

## **A. Construction of DABStep-Research Benchmark** 

Existing data science benchmarks typically focus only on evaluating the ability of LLMs to solve specific tasks. However, with the rise of deep research, there is an urgent need for a benchmark that assesses LLMs’ capabilities in data-oriented deep research, which ask LLMs to conduct data research and generate research reports based on given instructions and data sources.


key: chroma:document
string_value:
**Construction.** To this end, we constructed DABStep-Research, which is built upon the data sources proposed in DABStep (Egg et al., 2025). DABStep-Research consists of 100 tasks divided into five categories: data preparation, data analysis, data insight, report generation, and open-ended data research. In particular, tasks under the “report generation” category specify detailed report formats in the instruction, such as title, outline and specific requirements, thereby evaluating how well LLMs can follow instructions when generating research reports. The “open-ended data research” category involves fully open research tasks without any constraint on research direction or method. In addition to the instructions and data sources, we also provide a checklist to serve as a reference for scoring, helping evaluators determine whether the elements in a research report meet the given requirements. Figure 1 illustrates specific examples from DABStep-Research.


key: chroma:document
string_value:
**Evaluation.** We use the LLM-as-a-judge to evaluate LLM performance on DABStep-Research. Specifically, given the instruction, checklist, and the report generated by an LLM, we employ a state-of-the-art LLM as the evaluator to assign a score from 1 to 5 based on two aspects: content and format. The prompt used for the LLM-as-judge evaluation is shown below.


key: chroma:document
string_value:
Prompt of DABStep-Research Evaluation You are a data science evaluation assistant. Here’s a generated data science report based on the user instruction. Your task is to comprehensively evaluate the quality of the generated data science report, based on the provided user instruction [INSTRUCTION], a checklist offering reference points for an ideal report [CHECKLIST], and the generated report [REPORT]. You should assess the report across the following two dimensions, each scored on a scale from 1 (poor), 3 (Fair), 5 (excellent). Please use the detailed guidelines below to calibrate your evaluation: **– Content** : Is the report’s content helpful, comprehensive, and relevant to the task goal? **1 (Poor)** : Content is completely irrelevant, incorrect, or fails to reflect the given task. **2 (Weak)** : Mostly irrelevant or inaccurate; shows little understanding of the task or data. **3 (Fair)** : Partially relevant and somewhat useful, but incomplete, superficial, or missing several key


key: chroma:document
string_value:
(Weak)** : Mostly irrelevant or inaccurate; shows little understanding of the task or data. **3 (Fair)** : Partially relevant and somewhat useful, but incomplete, superficial, or missing several key aspects. **4 (Good)** : Relevant and generally helpful content that addresses the task goal with clear findings; minor gaps or shallow areas may remain. **5 (Excellent)** : Highly informative, comprehensive, and well-balanced content that fully and insightfully addresses the task goal. **– Format** : Is the report presented in a polished academic style? **1 (Poor)** : Disorganized or unprofessional presentation; difficult to follow, with major grammatical or formatting issues. **2 (Weak)** : Understandable but inconsistent in structure or tone; lacks clear formatting or proper academic expression, such as many short sentences and bullet points. **3 (Fair)** : Generally clear structure and readable style, though uneven in flow, tone, or academic polish. **4 (Good)** : Well-written and


key: chroma:document
string_value:
such as many short sentences and bullet points. **3 (Fair)** : Generally clear structure and readable style, though uneven in flow, tone, or academic polish. **4 (Good)** : Well-written and professionally presented in an academic style; clear organization and formatting with only minor imperfections. **5 (Excellent)** : Polished, fluent, and professional presentation; precise structure, coherent tone, and excellent readability throughout. **[INSTRUCTION]** : {instruction} **[CHECKLIST]** : {checklist} **[REPORT]** : {report} Directly return your evaluation in the following JSON format: ‘‘‘json { "Content": <score>, "Format": <score>, } ‘‘‘


key: chroma:document
string_value:
--- end of page.page_number=16 ---

**==> picture [297 x 9] intentionally omitted <==**

**----- Start of picture text -----**<br>
DeepAnalyze: Agentic Large Language Models for Autonomous Data Science<br>**----- End of picture text -----**<br>


**==> picture [485 x 159] intentionally omitted <==**


key: chroma:document
string_value:
Instruction Checklist Instruction Checklist<br>PreparationData Create a comprehensive data preparation report for optimizing payment processing fee calculations. The report must analyze the provided datasets (payments, merchant data, fees, acquirer countries, MCC codes) to identify and resolve data quality issues that would prevent accurate fee computation. Specifically, investigate data completeness, consistency across sources, and proper categorization required for applying the fee rules defined in the manual.  The research report must include:- Assessment of data completeness and identification of missing critical fields (e.g., merchant_category_code in payments, acquirer country details)- Validation of categorical data consistency against reference tables (e.g., ACI codes, MCC codes, account types)- Strategy for merging transaction data with merchant metadata and fee rules- Plan for handling null values in fee rule conditions as specified in the manual Generate a comprehensive


key: chroma:document
string_value:
account types)- Strategy for merging transaction data with merchant metadata and fee rules- Plan for handling null values in fee rule conditions as specified in the manual Generate a comprehensive data report analyzing the payment processing efficiency and cost optimization opportunities for the provided merchant data. The report must follow this exact structure and include all specified sections:# Merchant Payment Processing Analysis Report## 1. Executive Summary The research report must include:- Title exactly as "Merchant Payment Processing Analysis Report"- All 10 main sections as specified in the structure- Executive Summary with key findings and recommendations- Merchant Profile Analysis for each merchant in the dataset<br>The report should propose a cleaned, merged dataset ready for analysis and include a step-by-step plan for calculating the exact fee for each transaction based on merchant characteristics, transaction details, and the rules in fees.json. - Methodology for


key: chroma:document
string_value:
for analysis and include a step-by-step plan for calculating the exact fee for each transaction based on merchant characteristics, transaction details, and the rules in fees.json. - Methodology for calculating the final fee (fixed_amount + rate * transaction_value / 10000) for a sample of transactions- Summary of key data issues found and their impact on fee calculation accuracy ### 1.1 Key Findings Overview### 1.2 Critical Recommendations### 1.3 Financial Impact Assessment - Transaction volume and value analysis with numerical evidence- Fee structure optimization analysis referencing specific fee rules<br>## 2. Merchant Profile Analysis - Fraud rate assessment with current levels and patterns<br>Instruction Checklist ### 2.1 Account Type Classification### 2.2 Merchant Category Code (MCC) Analysis - Strategic recommendations categorized by implementation timeline<br>AnalysisData Analyze the payment transactions dataset to identify cost optimization opportunities for merchants by


key: chroma:document
string_value:
(MCC) Analysis - Strategic recommendations categorized by implementation timeline<br>AnalysisData Analyze the payment transactions dataset to identify cost optimization opportunities for merchants by examining fee structures, transaction patterns, and fraud metrics. Produce a comprehensive data science report that evaluates how different merchant characteristics (account type, MCC, volume), transaction attributes (ACI, card scheme, cross-border status), and fraud levels impact processing costs. The report should provide actionable recommendations for each merchant to minimize fees while maintaining security and conversion rates. The research report should include:- Analysis of fee distribution across different card schemes, ACIs, and merchant categories- Evaluation of cross-border vs domestic transaction costs and volume patterns- Fraud rate analysis and its correlation with processing costs- Monthly volume patterns and their impact on fee optimization- Merchant-specific cost analysis


key: chroma:document
string_value:
transaction costs and volume patterns- Fraud rate analysis and its correlation with processing costs- Monthly volume patterns and their impact on fee optimization- Merchant-specific cost analysis and optimization recommendations- Identification of transaction downgrade risks and prevention strategies- Summary of key cost drivers and prioritized optimization opportunities GenerationReport ### 2.3 Acquirer Network Configuration### 2.4 Capture Delay Patterns## 3. Transaction Performance Analysis### 3.1 Volume and Value Distribution#### 3.1.1 Transaction Volume by Merchant#### 3.1.2 Average Transaction Value Analysis#### 3.1.3 Temporal Patterns (Hourly/Daily/Seasonal)### 3.2 Authorization Characteristics - Implementation roadmap with priority sequencing- Conclusion summarizing opportunities and expected impact- Appendices with methodology and references- Clear section numbering following the hierarchical structure- Data-supported recommendations with specific numerical evidence-


key: chroma:document
string_value:
and expected impact- Appendices with methodology and references- Clear section numbering following the hierarchical structure- Data-supported recommendations with specific numerical evidence- Comparative analysis between different merchants<br>#### 3.2.1 ACI Distribution Analysis - Cross-references to fee rules and merchant category codes<br>Instruction Checklist #### 3.2.2 Card Scheme Performance#### 3.2.3 Credit vs Debit Transaction Patterns<br>InsightData Generate a comprehensive data insight report analyzing the payment transaction data to identify cost optimization opportunities for merchants. The report should analyze how different transaction characteristics (ACI, card scheme, merchant category, volume, fraud levels, etc.) impact processing fees and provide actionable recommendations for minimizing costs while maintaining security and conversion rates. Include specific analysis of how merchants can optimize their  The research report should include:- Analysis of fee structures


key: chroma:document
string_value:
for minimizing costs while maintaining security and conversion rates. Include specific analysis of how merchants can optimize their  The research report should include:- Analysis of fee structures across different transaction characteristics (ACI, card scheme, etc.)- Identification of highest cost drivers in the current payment processing- Comparison of domestic vs international transaction costs and optimization opportunities- Fraud rate impact analysis on processing fees and cost-benefit of prevention measures- Volume-based pricing analysis and recommendations for volume optimization- Merchant-specific cost optimization recommendations based on their business profiles omitted to save space… ## 10. Appendices### 10.1 Data Sources and Methodology### 10.2 Key Term Definitions### 10.3 Reference Tables<br>fee structures by adjusting transaction routing, authentication methods, and fraud prevention strategies based on their business profiles. - Actionable strategies for minimizing


key: chroma:document
string_value:
Reference Tables<br>fee structures by adjusting transaction routing, authentication methods, and fraud prevention strategies based on their business profiles. - Actionable strategies for minimizing downgrades and avoiding penalty fees- Cost-benefit analysis of different authentication methods and transaction routing options Format Requirements - Use clear, actionable business language:<br>- Include specific numerical evidence for all recommendations<br>Data ResearchOpen-ended Conduct open-ended data science research, finally output a research report.Instruction N/A Checklist - Provide merchant-specific analysis for each entity in the dataset- Cross-reference findings with the fee rules in fees.json- Include comparative analysis between merchants- All recommendations must be supported by data evidence<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
_Figure 9._ Cases in the constructed DABStep-Research benchmark, including data preparation, data analysis, data insight, report generation, and open-ended data research. 

## **B. Keyword-guided Reasoning Trajectory Synthesis** 

We present examples of Keyword-guided Reasoning Trajectory Synthesis in Figure 10. Specifically, the “Question” and Original Response are taken from existing TableQA datasets. 

In the distillation step, we employ SOTA closed-source LLMs as teacher models to extract their reasoning trajectories, which is the most common way used in current data synthesis methods. However, such methods are more suitable for general reasoning processes. Since SOTA closed-source LLMs have not been specifically trained on domains like data science (e.g., structured data understanding), their reasoning trajectories tend to overlook the provided data.


key: chroma:document
string_value:
Therefore, we introduce a refinement step to enhance the reasoning trajectory’s focus on structured data by inserting reasoning keywords that guide the reasoning process toward structured data understanding. Specifically, in the example shown in Figure 10, we sample three reasoning keywords (“What happens at the boundaries?”, “Let’s review the prior reasoning”, and “Let’s take a closer look at the table”) and ask the teacher model to refine its reasoning trajectory based on these keywords. We observe that the final “refinement” results exhibits a significantly stronger emphasis on repeated examination and reflection on structured data, thereby improving the overall quality of the reasoning trajectory. Overall, the proposed keyword-guided refinement is a useful data synthesis technique that can also be applied to the data synthesis of other complex tasks. 

## **C. Cases**


key: chroma:document
string_value:
## **C. Cases** 

In Figure 11, Figure 12, Figure 13, Figure 14, and Figure 15, we demonstrate a series of autonomous data science cases, covering the entire pipeline from data sources to analyst-grade research reports. These cases include data preparation, data analysis, data insight extraction, report generation under specific constraints, and fully open-ended data research. Compared with previous closed-source LLMs and tool-calling frameworks, DeepAnalyze can produce higher-quality, analyst-level reports, exhibiting a stronger ability for autonomous data research. 

--- end of page.page_number=17 ---


key: chroma:document
string_value:
|**Stage**|||**Reasoning Trajectory**||
|---|---|---|---|---|
|||Below is an instruction that describes a question answering task in the finance domain, provided with an input table and relevant text as context.|||
|||Please answer the given question based on the table and text.|||
|||### Instruction|||
|||Given a table and a list of texts in the following, what is the answer to the question?<br>### Table:|||
|||| Dry Bulk Vessel Operations for the Year Ended December 31, 2019 | Logistics Business for the Year Ended December 31, 2019 | Total for the Year Ended December 31, 2019|||
|||Revenue | $254,178 | $228,271 | $482,449|||
|||Administrative fee revenue from affiliates | 16,991 | — | 16,991|||
|||Interest income | 9,610 | 1,052 | 10,662|||
|||Interest expense and finance cost | (92,948) | (40,531) | (133,479)|||
|||Depreciation and amortization | (52,288) | (29,435) | (81,723)|||
|||Equity in net (losses)/earnings of affiliated companies | (9,185) | — | (9,185)|||


key: chroma:document
string_value:
|||Depreciation and amortization | (52,288) | (29,435) | (81,723)|||
|||Equity in net (losses)/earnings of affiliated companies | (9,185) | — | (9,185)|||
|||Net (loss)/ income attributable to Navios Holdings common | | ||||
|||stockholders | (212,623) | 20,513 | (192,110)|||
|||Total assets | 1,511,517 | 631,338 | 2,142,855|||
|||Goodwill | 56,240 | 104,096 | 160,336|||
|**Question**||Capital expenditures | (36,628) | (7,943) | (44,571)<br>Investment in affiliates | 64,352 | — | 64,352|||
|||Cash and cash equivalents | 32,386 | 45,605 | 77,991|||
|||Restricted cash | 736 | — | 736|||
|||Long-term debt, net (including current and noncurrent portion) | $1,048,318 ||$514,929 | 1,563,247||
|||### Text:|||
|||Paragraph 1: NAVIOS MARITIME HOLDINGS INC. NOTES TO THE CONSOLIDATED FINANCIAL STATEMENTS (Expressed in thousands of U.S. dollars — except share data)|||
|||Paragraph 2: NOTE 19: SEGMENT INFORMATION|||


key: chroma:document
string_value:
|||Paragraph 2: NOTE 19: SEGMENT INFORMATION|||
|||Paragraph 3: The Company through August 30, 2019 had three reportable segments from which it derived its revenues: Dry Bulk Vessel Operations, Logistics Business and Containers Business. The Containers Business became a reportable segment as a result of the consolidation of Navios Containers since November||30, 2018 (date of obtaining control) (see also Note 3). Following|
|||the reclassification of the results of Navios Containers as discontinued operations (see also Note 3), the Company currently has two reportable segments from which it derives its revenues: Dry Bulk Vessel Operations and Logistics. The reportable segments reflect the internal organization of the Company and are strategic businesses that offer different products and services. The Dry|||


key: chroma:document
string_value:
|||Bulk Vessel Operations consists of the transportation and handling of bulk cargoes through the ownership, operation, and trading of vessels. The Logistics Business consists of operating ports and transfer station terminals, handling of vessels, barges and pushboats as well as upriver transport facilities in the Hidrovia region.|||
|||Paragraph 4: The Company measures segment performance based on net income/ (loss) attributable to Navios Holdings common stockholders. Inter-segment sales and transfers are not significant and have been eliminated and are not included in the following tables. Summarized financial information concerning each of the Company’s reportable segments is as follows:|||
|||### Question:|||
|||What was the difference in revenue between dry bulk vessel operations and logistics business?|||
|||### Response|||
|||Your response should end with `Answer: xxx` (answer to the question).|||
|||Now, answer the question step by step:|||
|**Original**||Answer: 25,907|||


key: chroma:document
string_value:
|||### Response|||
|||Your response should end with `Answer: xxx` (answer to the question).|||
|||Now, answer the question step by step:|||
|**Original**||Answer: 25,907|||
|||<think>|||
|||Alright, let's tackle this finance question step by step. The user wants to know the revenue difference between dry bulk vessel operations and logistics business for Navios Holdings in 2019.|||
|||First, I'll locate the revenue figures in the provided table. Scanning the first data row under "Dry Bulk Vessel Operations" shows $254,178, while "Logistics Business" shows $228,271. Both are clearly labeled under the 2019 year-end columns.|||
|||The question specifically asks for the difference, which implies subtraction. Calculating: $254,178 minus $228,271 equals $25,907.|||


key: chroma:document
string_value:
|||The question specifically asks for the difference, which implies subtraction. Calculating: $254,178 minus $228,271 equals $25,907.|||
|||Now, should I consider any context from the text? Paragraph 3 confirms these are the two active reportable segments after containers were reclassified as discontinued operations. Paragraph 4 emphasizes they're measured by net income/loss (irrelevant here) but crucially states inter-segment transactions are eliminated||- meaning these revenues aren't inflated by internal transfers.|
|||No currency or unit conversions are needed since everything's in thousands of USD. The math is straightforward: no percentages or ratios required, just absolute difference.|||
|||Double-checking: table headers align with the question's timeframe, and no other revenue lines (like administrative fees) are mixed in because the question isolates "revenue" for each segment.|||
|||Final verification: $254,178 - $228,271 = $25,907. This seems solid.<br></think>|||


key: chroma:document
string_value:
|||Final verification: $254,178 - $228,271 = $25,907. This seems solid.<br></think>|||
|**Distillation**||To determine the revenue difference between dry bulk vessel operations and logistics business:|||
|||1. **Identify relevant revenue figures** from the table:|||
|||- Dry Bulk Vessel Operations revenue: **$254,178**|||
|||- Logistics Business revenue: **$228,271**|||
|||2. **Calculate the difference**:|||
|||Subtract Logistics revenue from Dry Bulk revenue:|||
|||$254,178 - $228,271 = **$25,907**|||
|||3. **Verify context** (Text Paragraphs 3-4):|||
|||- The segments are distinct, with no significant inter-segment transactions.|||
|||- Revenues are explicitly reported per segment in the table.|||
|||- The calculation aligns with the question’s focus on standalone segment revenues.|||
|||Answer: $25,907|||
|||<Analyze>|||


key: chroma:document
string_value:
|||- Revenues are explicitly reported per segment in the table.|||
|||- The calculation aligns with the question’s focus on standalone segment revenues.|||
|||Answer: $25,907|||
|||<Analyze>|||
|||Let's begin by clearly defining the problem at hand: the question asks for the difference in revenue between the Dry Bulk Vessel Operations segment and the Logistics Business segment for the year ended December 31, 2019. To solve this, I need to follow a structured approach: first, confirm the relevant segments and their revenue metrics, then locate the specific numerical values in the|||
|||provided table, verify that these values are accurate and unadjusted by inter-segment transactions, perform the subtraction to find the difference, and finally cross-check with the context in the text to ensure no critical details are missed.|||


key: chroma:document
string_value:
|||First, I should confirm which segments are being compared. The text in Paragraph 3 notes that after August 30, 2019, the Company has two reportable segments: Dry Bulk Vessel Operations and Logistics Business (since the Containers Business was reclassified as discontinued operations). This aligns with the table, which only lists these two segments under the "Year Ended December|||
|||31, 2019" columns, so these are the correct segments to focus on.|||
|||Next, the key metric here is "revenue." I need to ensure I am selecting the right<br></Analyze><br><Understand>|line item in the table. The table has multiple rows, including "Administrative fee revenue from affiliates," "Interest income," etc., but the question specifically asks for "revenue," which is the top-line item.**Let's take a closer look at the table**|<br>to identify the "Revenue" row.|


key: chroma:document
string_value:
|||The table is titled "Dry Bulk Vessel Operations for the Year Ended December 31, 2019," "Logistics Business for the Year Ended December 31, 2019," and "Total for the Year Ended December 31, 2019." The first data row in the table is labeled "Revenue." Under this row:|||
|||- The value for Dry Bulk Vessel Operations is $254,178.|||
|||- The value for Logistics Business is $228,271.|||
|||- The "Total" column for Revenue is $482,449, which is the sum of the two segment revenues ($254,178 + $228,271 = $482,449), confirming that these are the primary revenue figures for the segments and not adjusted by other line items.|||


key: chroma:document
string_value:
|||It is critical to note that other rows like "Administrative fee revenue from affiliates" ($16,991 for Dry Bulk) or "Interest income" ($9,610 for Dry Bulk, $1,052 for Logistics) are separate from the core "Revenue" line item. The question explicitly asks for "revenue," so these secondary income streams are irrelevant here. The table structure clearly separates "Revenue" as the first row,|||
|||making it the correct metric to use.|||
|||</Understand>|||
|||<Analyze>|||
|||Now that I have identified the relevant revenue figures for both segments, the next step is to calculate the difference between them. The question asks for the "difference," which in financial contexts typically refers to the absolute or directional variance between two values. Since Dry Bulk Vessel Operations revenue ($254,178) is larger than Logistics Business revenue ($228,271), the|||
|||difference will be Dry Bulk revenue minus Logistics revenue.|||


key: chroma:document
string_value:
|||difference will be Dry Bulk revenue minus Logistics revenue.|||
|||Before proceeding with the calculation, I should verify that there are no adjustments needed to these revenue figures. The text in Paragraph 4 states: "Inter-segment sales and transfers are not significant and have been eliminated and are not included in the following tables." This is crucial because if there were significant inter-segment transactions, the reported revenues might include|||
|||internal sales, which would need to be excluded to get the true standalone revenue of each segment. However, the text explicitly confirms that such transactions are eliminated, so the revenues in the table represent the external, standalone revenue of each segment. This ensures the numbers are accurate for comparison.|||
|||Let's now perform the numerical subtraction step by step to avoid errors.|||
|||</Analyze>|||
|||<Understand>|||


key: chroma:document
string_value:
|||Let's now perform the numerical subtraction step by step to avoid errors.|||
|||</Analyze>|||
|||<Understand>|||
|**Refinement**||To calculate the difference:<br>Dry Bulk Vessel Operations Revenue: $254,178<br>Logistics Business Revenue: $228,271|||
|**Insert Key Reasoning**|||||
|**Words:**||Subtract Logistics revenue from Dry Bulk revenue:<br>$254,178 - $228,271|||
|**1. What happens at the**|||||
|**boundaries?**||Let’s break down the subtraction:<br>- 254,178 - 228,271 = (254,000 - 228,000) + (178 - 271)|||
|**2. Let's review the prior**||- 254,000 - 228,000 = 26,000|||
|**reasoning**||- 178 - 271 = -93<br>- Total: 26,000 - 93 = 25,907|||
|**3. Let's take a closer look**|||||
|**at the table**||Thus, the result is $25,907.|||
|||To confirm accuracy, we can also check by adding the difference to Logistics revenue to see if we get back to Dry Bulk revenue: $228,271 + $25,907 = $254,178, which matches. This cross-verification ensures the subtraction was performed correctly.|||


key: chroma:document
string_value:
|||</Understand>|||
|||<Analyze>|||
|||**Let's review the prior reasoning**<br> to ensure no steps were overlooked. We started by confirming the relevant segments (Dry Bulk and Logistics) based on the text, which eliminated the discontinued Containers segment. We then identified the "Revenue" row in the table as the correct metric, distinguishing it from other income lines like administrative fees or interest income. We|||
|||extracted the values ($254,178 and $228,271) and verified they sum to the total|revenue, confirming their validity. We checked for inter-segment transactions, which the text stated are eliminated, so no adjustments were needed. Finally, we calculated the difference via subtraction and cross-verified the result.||


key: chroma:document
string_value:
|||**What happens at the boundaries?**<br>For example, could the "Total" revenue include other segments? The text in Paragraph 3 clarifies that after August 30, 2019, there are only two reportable segments, and the table only lists these two, so the total revenue is indeed the sum of the two. There are no other segments or hidden revenues to consider.|||
|||Is there any chance the units are inconsistent? The text in Paragraph 1 specifies|that the financial statements are "Expressed in thousands of U.S. dollars," and both revenue figures are in the same units (thousands of USD), so no unit conversion is necessary. The calculated difference of $25,907 is therefore in thousands|of USD, which aligns with the question’s context.|
|||All checks confirm the reasoning is sound, and the calculation is accurate. The|final step is to present the answer.||
|||</Analyze>|||
|||<Answer>|||
|||To determine the revenue difference between dry bulk vessel operations and logistics business:|||


key: chroma:document
string_value:
|||</Analyze>|||
|||<Answer>|||
|||To determine the revenue difference between dry bulk vessel operations and logistics business:|||
|||1. **Identify relevant revenue figures** from the table:|||
|||- Dry Bulk Vessel Operations revenue: **$254,178**|||
|||- Logistics Business revenue: **$228,271**|||
|||2. **Calculate the difference**:|||
|||Subtract Logistics revenue from Dry Bulk revenue:|||
|||$254,178 - $228,271 = **$25,907**|||
|||3. **Verify context** (Text Paragraphs 3-4):|||
|||- The segments are distinct, with no significant inter-segment transactions.|||
|||- Revenues are explicitly reported per segment in the table.|||
|||- The calculation aligns with the question’s focus on standalone segment revenues.|||
|||Answer: $25,907|||
|||</Answer>|||
||||_Figure 10._ Example of reasoning trajectory synthesis.||


key: chroma:document
string_value:
|Analyze the payment processing<br>dataset to identify and resolve data<br>quality issues that impact fee<br>calculation accuracy. Prepare a<br>comprehensive data preparation<br>report that documents data<br>inconsistencies, implements<br>cleaning strategies for merchant<br>categorization, transaction<br>characteristics, and country<br>mappings, and evaluates how these<br>preparations affect the reliability of<br>fee estimation models. The report<br>should include specific<br>recommendations for optimizing<br>data collection processes to<br>minimize processing costs.|# Comprehensive Data Preparation Report: Payment Processing Fee Analysis<br>## Executive Summary<br>This report analyzes payment processing datasets to identify data quality issues impacting fee<br>calculation accuracy. Through examination of 7 datasets (30 merchants, 1,000 fee rules, and<br>synthetic transaction data), we identified key data quality factors affecting fee estimation<br>reliability and recommend process


key: chroma:document
string_value:
examination of 7 datasets (30 merchants, 1,000 fee rules, and<br>synthetic transaction data), we identified key data quality factors affecting fee estimation<br>reliability and recommend process improvements to minimize processing costs.<br>## 1. Data Quality Assessment<br>### 1.1 Merchant Categorization<br>- **Completeness**: 100% complete merchant-category mappings across all datasets<br>- **Consistency**: Merchant names consistently represented across merchant_data.json and<br>payments.csv<br>- **Coverage**: 7 unique merchant category codes among 30 merchants<br>### 1.2 Geographic Data<br>- **Acquirer Countries**: Complete mapping of 8 acquirers across 5 countries<br>- **Transaction Data**: Requires validation against merchant locations<br>### 1.3 Fee Rule Structure<br>- **Completeness**: 90% complete is_credit values in fee rules<br>- **Complexity**: 1,000 rules with 12 attributes each averaging 4.2 conditional constraints<br>## 2. Data Preparation Steps<br>### 2.1 Cleaning


key: chroma:document
string_value:
90% complete is_credit values in fee rules<br>- **Complexity**: 1,000 rules with 12 attributes each averaging 4.2 conditional constraints<br>## 2. Data Preparation Steps<br>### 2.1 Cleaning Strategies<br>1. **Merchant Matching**:<br>- Developed fuzzy matching algorithm for merchant name normalization (92% match rate)<br>- Created crosswalk table for variant merchant names<br>2. **Fee Rule Validation**:<br>- Flagged 100 rules with missing is_credit values for review<br>- Standardized merchant category code lists in fee rules<br>3. **Geographic Resolution**:<br>- Enhanced acquirer country mapping with IP geolocation fallback<br>### 2.2 Quality Metrics<br>| Metric | Before Cleaning | After Cleaning |<br>|--------|-----------------|-----------------|<br>| Missing Merchant Matches | 18% | 2% |<br>| Inconsistent Categories | 12% | 0% |<br>| Unclassified Transactions | 9% | 1% |<br>## 3. Impact on Fee Estimation<br>### 3.1 Accuracy Improvement<br>- **Current**:±8.2% fee calculation error due


key: chroma:document
string_value:
Inconsistent Categories | 12% | 0% |<br>| Unclassified Transactions | 9% | 1% |<br>## 3. Impact on Fee Estimation<br>### 3.1 Accuracy Improvement<br>- **Current**:±8.2% fee calculation error due to categorization issues<br>- **After Cleaning**:±1.3% error margin<br>### 3.2 Processing Cost Impact<br>| Factor | Current Cost | Optimized Cost | Reduction |<br>|---------|-------------|----------------|-----------|<br>| Fraudulent Fee Disputes | $12,400/month | $2,800/month | 77% |<br>| Incorrect Fees | $8,700/month | $1,200/month | 86% |<br>## 4. Recommendations<br>### 4.1 Data Collection Improvements<br>1. **Real-Time Merchant Categorization**:<br>- Implement AI-powered category suggestion during onboarding<br>- Estimated 30% reduction in dispute resolution costs<br>2. **Enhanced Transaction Attributes**:<br>- Collect merchant location granularity (city/region)<br>- Add product/service categories<br>3. **Automated Data Validation**:<br>- Implement real-time rule validation during


key: chroma:document
string_value:
Attributes**:<br>- Collect merchant location granularity (city/region)<br>- Add product/service categories<br>3. **Automated Data Validation**:<br>- Implement real-time rule validation during transaction processing<br>- Estimated 25% reduction in manual reconciliation<br>### 4.2 Process Optimization<br>1. **Batch Processing Windows**:<br>- Shift non-essential processing to off-peak hours<br>- Potential 15% energy cost reduction<br>2. **Predictive Fee Allocation**:<br>- Implement machine learning for fee estimation<br>- Projected 18% improvement in accuracy<br>## 5. Conclusion<br>This analysis demonstrates that systematic data preparation can significantly improve fee<br>calculation accuracy while reducing operational costs. The implemented solutions provide a<br>foundation for continuous improvement through ongoing monitoring of the data quality metrics<br>and processing efficiency gains.<br>Appendix: Complete dataset schemas and cleaning code available upon request.|Based on the


key: chroma:document
string_value:
improvement through ongoing monitoring of the data quality metrics<br>and processing efficiency gains.<br>Appendix: Complete dataset schemas and cleaning code available upon request.|Based on the initial analysis of the payment processing dataset, I've identified several critical data<br>quality issues. Here's a comprehensive report structured according to your requirements:<br>### 1. Data Quality Issues and Inconsistencies<br>- **Missing Data**:<br>- `acquirer_country`: 12,543 missing values (5.4% of transactions)<br>- `merchant_category_code`: 8,217 missing values (3.6% of transactions)<br>- `transaction_currency`: 1,205 missing values (0.5% of transactions)<br>- **Invalid References**:<br>- Merchant Category Codes: 3,581 transactions (1.6%) contain invalid MCCs not present in the<br>reference table<br>- Acquirer Countries: 2,894 transactions (1.3%) reference non-existent country codes<br>- **Data Distribution Anomalies**:<br>- 95% of transactions concentrated in just 15 MCC


key: chroma:document
string_value:
table<br>- Acquirer Countries: 2,894 transactions (1.3%) reference non-existent country codes<br>- **Data Distribution Anomalies**:<br>- 95% of transactions concentrated in just 15 MCC categories<br>- 87% of transactions originate from only 5 countries<br>- Transaction amounts show significant right-skewness (mean = $152, max = $32,500)<br>### 2. Data Cleaning Strategies<br>**A. Merchant Categorization**<br>```json<br>{"code":"# Handle invalid/missing MCCs\<br>valid_mcc = set(merchant_categories['mcc'])\<br>payments['mcc_valid'] = payments['merchant_category_code'].isin(valid_mcc)\<br>payments.loc[~payments['mcc_valid'], 'merchant_category_code'] = 9999 # Unknown category\<br>payments['merchant_category_code'].fillna(9999, inplace=True)"}<br>```<br>**B. Country Mapping**<br>```json<br>{"code":"# Clean country references\<br>valid_countries = set(acquirer_countries['country_code'])\<br>payments['country_valid'] =


key: chroma:document
string_value:
inplace=True)"}<br>```<br>**B. Country Mapping**<br>```json<br>{"code":"# Clean country references\<br>valid_countries = set(acquirer_countries['country_code'])\<br>payments['country_valid'] = payments['acquirer_country'].isin(valid_countries)\<br>payments.loc[~payments['country_valid'], 'acquirer_country'] = 'ZZ' # Unknown country\<br>payments['acquirer_country'].fillna('ZZ', inplace=True)\<br>\<br># Create region mapping\<br>region_map = acquirer_countries.set_index('country_code')['region'].to_dict()\<br>payments['acquirer_region'] = payments['acquirer_country'].map(region_map)\<br>payments['acquirer_region'].fillna('Unknown', inplace=True)"}<br>```<br>**C. Transaction Characteristics**<br>```json<br>{"code":"# Handle missing currencies\<br>payments['transaction_currency'].fillna('USD', inplace=True)\<br>\<br># Detect and cap outliers\<br>Q1 = payments['transaction_amount'].quantile(0.25)\<br>Q3 = payments['transaction_amount'].quantile(0.75)\<br>IQR = Q3 - Q1\<br>upper_bound = Q3


key: chroma:document
string_value:
inplace=True)\<br>\<br># Detect and cap outliers\<br>Q1 = payments['transaction_amount'].quantile(0.25)\<br>Q3 = payments['transaction_amount'].quantile(0.75)\<br>IQR = Q3 - Q1\<br>upper_bound = Q3 + 3*IQR\<br>payments['transaction_amount'] = payments['transaction_amount'].clip(upper=upper_bound)"}<br>```<br>### 3. Impact on Fee Calculation Accuracy<br>- **Before Cleaning**:<br>- 7.9% of transactions (18,193) would have incorrect fee calculations due to invalid/missing<br>references<br>- Extreme transaction amounts would distort interchange fee calculations<br>- Regional fee variations couldn't be accurately applied<br>- **After Cleaning**:<br>- 100% reference integrity for MCCs and countries<br>- Outlier capping ensures stable fee calculations<br>- Region-based fee rules can be precisely applied<br>- Estimated fee calculation error rate reduced from±8.2% to±0.9%<br>### 4. Recommendations for Data Collection Optimization<br>1. **Validation at Ingestion**:<br>- Implement MCC and


key: chroma:document
string_value:
applied<br>- Estimated fee calculation error rate reduced from±8.2% to±0.9%<br>### 4. Recommendations for Data Collection Optimization<br>1. **Validation at Ingestion**:<br>- Implement MCC and country code validation during transaction recording<br>- Add mandatory field checks for `acquirer_country` and `merchant_category_code`<br>2. **Automated Quality Monitoring**:<br>- Create daily data quality dashboards tracking:<br>- Missing value rates per critical field<br>- Reference table match rates<br>- Transaction amount distribution shifts<br>3. **Reference Data Management**:<br>- Establish monthly review process for MCC and country code updates<br>- Implement version control for fee structure documents<br>4. **Standardization**:<br>- Enforce ISO standards for country codes (ISO 3166) and currencies (ISO 4217)<br>- Develop MCC taxonomy with clear category definitions<br>These improvements will reduce processing costs by minimizing reconciliation efforts, preventing<br>fee


key: chroma:document
string_value:
3166) and currencies (ISO 4217)<br>- Develop MCC taxonomy with clear category definitions<br>These improvements will reduce processing costs by minimizing reconciliation efforts, preventing<br>fee miscalculations, and enabling more accurate financial forecasting. The cleaned dataset now<br>supports reliable fee modeling with complete reference integrity across all critical dimensions.|Below is the final data preparation report:<br>---------------------------------------------------------<br>Data Preparation Report for Payment Processing Dataset<br>---------------------------------------------------------<br>• Data files were loaded successfully:<br>- acquirer_countries.csv loaded successfully with the acquirer country mappings.<br>- fees.json loaded as fee definitions for fee calculation.<br>- merchant_category_codes.csv loaded to support standardized merchant categorization.<br>- merchant_data.json loaded and converted to a DataFrame, representing additional


key: chroma:document
string_value:
for fee calculation.<br>- merchant_category_codes.csv loaded to support standardized merchant categorization.<br>- merchant_data.json loaded and converted to a DataFrame, representing additional merchant<br>metadata.<br>- payments.csv loaded (largest dataset) with comprehensive transaction records.<br>• Preliminary analysis of payments data exposed several data quality issues:<br>1. Missing Values & Duplicates:<br>- A summary of missing values per field was computed. Several columns had missing entries that<br>require further imputation.<br>- Duplicate records were identified, indicating the need for deduplication.<br>2. Merchant Categorization Inconsistency:<br>- Discrepancies were noted between merchant categories in payments.csv and those in<br>merchant_category_codes.csv. This inconsistency could cause improper fee calculations if similar<br>items are miscategorized.<br>3. Transaction Characteristics:<br>- Transaction amounts display potential outliers and missing data. These


key: chroma:document
string_value:
could cause improper fee calculations if similar<br>items are miscategorized.<br>3. Transaction Characteristics:<br>- Transaction amounts display potential outliers and missing data. These issues could bias fee<br>estimates if not addressed.<br>4. Country Mappings:<br>- A cross-check between acquirer_country from payments and acquirer_countries.csv revealed<br>potential mismatches that might affect regional fee-related rules.<br>• Cleaning Strategies Implemented:<br>- Merchant Categorization Standardization:<br>• Use merchant_category_codes.csv to remap and enforce consistency via string normalization<br>(trimming, lowercasing, etc.).<br>- Transaction Characteristics:<br>• Implement imputation strategies (e.g., median imputation) for missing transaction_amount<br>values.<br>• Employ domain analysis to identify and trim extreme outliers to avoid distortion.<br>- Country Mappings:<br>• Merge payments data with acquirer_countries.csv to validate and correct country identifiers.<br>•


key: chroma:document
string_value:
analysis to identify and trim extreme outliers to avoid distortion.<br>- Country Mappings:<br>• Merge payments data with acquirer_countries.csv to validate and correct country identifiers.<br>• Impact on Fee Estimation Models:<br>• These data cleaning strategies ensure fee calculation models operate on properly categorized and<br>validated data.<br>• Improvements such as deduplication, imputation, and outlier trimming help in reducing biases,<br>leading to more reliable fee estimations.<br>• Recommendations for Optimizing Data Collection Processes:<br>1. Standardize Data Entry:<br>- Enforce uniform data entry formats using dropdowns or controlled vocabularies for merchant<br>categories and country codes.<br>2. Real-time Validation:<br>- Integrate robust validation rules at data entry (e.g., for transaction amounts, dates, and fee<br>parameters) to catch errors early.<br>3. Regular Automated Audits:<br>- Automate periodic audits to monitor merging processes across different source


key: chroma:document
string_value:
(e.g., for transaction amounts, dates, and fee<br>parameters) to catch errors early.<br>3. Regular Automated Audits:<br>- Automate periodic audits to monitor merging processes across different source files (fees.json,<br>payments.csv, etc.) and quickly identify any discrepancies.<br>4. Comprehensive Data Dictionary:<br>- Maintain an up-to-date data dictionary linking all source files and field definitions, facilitating<br>smoother integrations and further reporting.<br>A detailed report has been saved as data_preparation_report.txt. Following these<br>recommendations will help minimize processing costs and improve the accuracy of fee estimation<br>models by ensuring higher data quality throughout the processing pipeline.<br>---------------------------------------------------------<br>End of Report<br>---------------------------------------------------------|


key: chroma:document
string_value:
_Figure 11._ A data preparation case of autonomous data science, from data sources to analyst-grade research reports. 

--- end of page.page_number=19 ---


key: chroma:document
string_value:
|Analyze the payment transaction<br>dataset to identify cost optimization<br>opportunities for merchants by<br>examining fee structures, fraud<br>patterns, and transaction<br>characteristics. Generate a<br>comprehensive data science report<br>that provides actionable<br>recommendations for minimizing<br>payment processing fees while<br>maintaining security and conversion<br>rates. The report should include<br>analysis of fee drivers, fraud risk<br>factors, acquirer selection strategies,<br>and specific optimization<br>recommendations tailored to<br>different merchant profiles.|# Comprehensive Analysis of Payment Processing Optimization Opportunities<br>## Executive Summary<br>This comprehensive analysis examines payment processing data from multiple European<br>merchants to identify cost optimization opportunities through detailed examination of fee<br>structures, fraud patterns, and transaction characteristics. The study leverages a dataset<br>comprising over 23,000 payment


key: chroma:document
string_value:
cost optimization opportunities through detailed examination of fee<br>structures, fraud patterns, and transaction characteristics. The study leverages a dataset<br>comprising over 23,000 payment transactions across 1,000 fee structures, 769 merchant categories,<br>and 8 acquirer banks. Our methodology combines advanced statistical analysis with business rule<br>optimization to identify actionable strategies for reducing payment processing costs while<br>maintaining security and conversion rates.<br>The analysis reveals that merchants can potentially reduce processing fees by 18-27% through<br>strategic account type selection, optimal acquirer selection, and transaction routing optimization.<br>Fraud prevention strategies can simultaneously reduce fraud-related fees by an estimated 35-42%<br>while maintaining current fraud levels. The findings are supported by detailed segmentation<br>analysis of 112 merchant profiles across 7 primary merchant categories.<br>## Merchant Profile


key: chroma:document
string_value:
35-42%<br>while maintaining current fraud levels. The findings are supported by detailed segmentation<br>analysis of 112 merchant profiles across 7 primary merchant categories.<br>## Merchant Profile Analysis and Segmentation<br>The merchant base comprises 112 distinct merchants categorized across 769 merchant category<br>codes, with the most prevalent categories being Membership Clubs (Sports, Recreation, Athletic),<br>Eating Places and Restaurants, and Book Stores. The merchant distribution shows a balanced<br>account type distribution with 34% Hospitality (H), 31% Franchise (F), 22% Retail (R), 9%<br>Digital (D), and 4% SaaS (S) accounts. This segmentation reveals significant variation in fee<br>structures across merchant types, with Hospitality merchants paying 18-22% higher processing<br>fees on average compared to Retail merchants.<br>Merchant acquirer relationships show 42% of merchants utilize 2+ acquirers, with an average of<br>1.7 acquirers per merchant. The most common


key: chroma:document
string_value:
on average compared to Retail merchants.<br>Merchant acquirer relationships show 42% of merchants utilize 2+ acquirers, with an average of<br>1.7 acquirers per merchant. The most common acquirer relationships involve Gringotts Bank<br>(GB) and The Savings and Loan Bank (US), handling 38% of total transaction volume. Merchant<br>category analysis reveals that Eating Places and Restaurants account for 22% of transaction<br>volume but only 15% of merchant count, indicating higher average transaction values in this<br>category.<br>The enhanced merchant profiles reveal that 68% of merchants fall into high-risk merchant<br>categories (MCCs 5812, 7997, 5942) which typically incur 23-31% higher fraud-related fees.<br>Merchant capture delay analysis shows that merchants with immediate capture delays experience<br>15-18% lower chargeback rates but 7-9% higher processing fees compared to manual capture<br>merchants.<br>## Fee Structure Optimization Opportunities<br>The fee structure analysis


key: chroma:document
string_value:
experience<br>15-18% lower chargeback rates but 7-9% higher processing fees compared to manual capture<br>merchants.<br>## Fee Structure Optimization Opportunities<br>The fee structure analysis reveals complex fee calculations combining fixed amounts (€0.01-€0.15)<br>and variable rates (5-19 basis points) across 1,000 fee rules. The most common fee structure<br>combines a €0.05 fixed amount with 12 basis points, applied to 38% of transactions. Merchant<br>category analysis shows that Hospitality merchants pay 23% higher fees than Retail merchants for<br>equivalent transaction values, primarily due to higher fraud risk premiums.<br>Account type analysis reveals that Digital (D) merchants pay 18-22% lower fees than Hospitality<br>(H) merchants for equivalent transaction values, despite similar fraud rates. The data shows that<br>switching from Hospitality to Retail account types could save merchants €12,000 annually for a<br>€100,000 volume merchant. Merchant category optimization


key: chroma:document
string_value:
similar fraud rates. The data shows that<br>switching from Hospitality to Retail account types could save merchants €12,000 annually for a<br>€100,000 volume merchant. Merchant category optimization opportunities vary from 12-18%<br>savings for Retail merchants to 23-27% for Digital merchants.<br>Acquirer selection analysis reveals that merchants using multiple acquirers can reduce fees by 8-<br>12% through optimal routing. The data shows that 62% of merchants could reduce fees by 5-8%<br>by shifting just 20% of volume to lower-fee acquirers. Fraud prevention strategies can reduce<br>fraud-related fees by 35-42% while maintaining current fraud levels through tokenization (12-15%<br>reduction) and 3D Secure (23-27% reduction).<br>## Transaction Pattern Analysis and Fraud Prevention<br>Transaction pattern analysis reveals that 68% of fraudulent transactions occur during off-peak<br>hours (2-5 AM local time) and from non-standard IP locations (72%). The data shows that<br>implementing


key: chroma:document
string_value:
pattern analysis reveals that 68% of fraudulent transactions occur during off-peak<br>hours (2-5 AM local time) and from non-standard IP locations (72%). The data shows that<br>implementing device fingerprinting could reduce fraud by 15-18% with minimal false positives (2-<br>3%). Authorization Characteristics Indicator (ACI) analysis shows that ACI B (Card Not Present -<br>3D Secure) transactions have 23-27% lower fraud rates but 12-15% higher processing fees than<br>ACI C (Tokenized card not present).<br>Merchant category fraud rates vary from 2.1% for Book Stores to 8.3% for Eating Places and<br>Restaurants. The data shows that implementing fraud prevention strategies can reduce fraud-<br>related fees by 35-42% while maintaining current fraud levels. The optimal fraud prevention<br>strategy combines tokenization (12-15% fraud reduction) with 3D Secure (23-27% reduction) and<br>device fingerprinting (15-18% reduction).<br>Transaction value analysis reveals that merchants accepting


key: chroma:document
string_value:
combines tokenization (12-15% fraud reduction) with 3D Secure (23-27% reduction) and<br>device fingerprinting (15-18% reduction).<br>Transaction value analysis reveals that merchants accepting €50-€100 transactions pay 18-22%<br>lower fees than those accepting >€500 transactions, suggesting optimal transaction splitting<br>strategies. The data shows that splitting €500 transactions into two €250 transactions could save<br>merchants 12-15% on processing fees while maintaining customer experience.<br>## Implementation Roadmap and Recommendations<br>The implementation roadmap suggests a phased approach to fee optimization, beginning with<br>account type optimization (Month 1-2, saving 8-12%), followed by acquirer optimization (Month<br>3-4, saving 5-8%), and concluding with fraud prevention implementation (Month 5-6, saving 35-<br>42%). The total potential savings for a €100,000 volume merchant could reach €18,000 annually<br>through comprehensive implementation.<br>The analysis


key: chroma:document
string_value:
implementation (Month 5-6, saving 35-<br>42%). The total potential savings for a €100,000 volume merchant could reach €18,000 annually<br>through comprehensive implementation.<br>The analysis identifies three primary optimization pathways: Account Type Optimization (12-<br>18% savings), Merchant Category Optimization (15-22% savings), and Fraud Prevention (35-42%<br>savings). The implementation requires minimal technical changes but requires strategic account<br>management and process changes. The payback period for fraud prevention technologies averages<br>6-9 months, with ROI exceeding 3:1 for most merchants.<br>The complete implementation roadmap includes detailed merchant-specific optimization plans<br>based on the segmentation analysis, with tailored strategies for Hospitality, Retail, and Digital<br>merchants. The analysis provides merchant-specific fee optimization calculators and scenario<br>modeling tools to project potential savings for each merchant profile.<br>##


key: chroma:document
string_value:
Retail, and Digital<br>merchants. The analysis provides merchant-specific fee optimization calculators and scenario<br>modeling tools to project potential savings for each merchant profile.<br>## Conclusion and Future Research Directions<br>This comprehensive analysis demonstrates that systematic payment processing optimization can<br>significantly reduce merchant costs while maintaining security and conversion rates. The findings<br>provide actionable insights for merchants, processors, and payment service providers to optimize<br>payment processing strategies. Future research directions include real-time fee optimization<br>algorithms and machine learning models for dynamic fee optimization based on transaction<br>characteristics.<br>The analysis methodology and findings provide a foundation for ongoing payment processing<br>optimization and continuous improvement. The demonstrated savings opportunities of 18-27%<br>for account type optimization, 15-22% for merchant category


key: chroma:document
string_value:
a foundation for ongoing payment processing<br>optimization and continuous improvement. The demonstrated savings opportunities of 18-27%<br>for account type optimization, 15-22% for merchant category optimization, and 35-42% for fraud<br>prevention collectively represent significant value creation for the payment processing ecosystem.|### Comprehensive Data Science Report: Payment Processing Optimization<br>#### 1. Executive Summary<br>Analysis of 23 million+ payment transactions reveals significant cost optimization opportunities<br>through strategic fee management, fraud reduction, and acquirer selection. Key findings:<br>- Average processing fee: 2.25% of transaction value<br>- Fraud rate: 0.97% overall with substantial variation by category<br>- Cross-border transactions carry 0.83% fee premium and 1.42% higher fraud risk<br>- Top 3 high-fee merchant categories: Digital Goods (3.8%), Travel Services (3.2%), Electronics<br>(2.9%)<br>- Top 3 high-fraud categories: Digital Goods


key: chroma:document
string_value:
fee premium and 1.42% higher fraud risk<br>- Top 3 high-fee merchant categories: Digital Goods (3.8%), Travel Services (3.2%), Electronics<br>(2.9%)<br>- Top 3 high-fraud categories: Digital Goods (4.2%), Gift Cards (3.8%), Luxury Goods (2.7%)<br>#### 2. Fee Structure Analysis<br>**Key drivers of payment processing fees:**<br>- **Merchant Category:** Digital goods merchants pay highest fees (3.8% average)<br>- **Transaction Size:** Smaller transactions (<$50) incur highest relative fees (3.1% vs 1.6% for<br>>$1000)<br>- **Card Type:** Premium cards (Platinum/Black) cost merchants 0.7-1.2% more than standard<br>cards<br>- **Cross-Border Premium:** International transactions cost 0.83% more than domestic<br>- **Acquirer Location:** Transactions processed through certain countries (CY, MT, LU) show<br>0.5-0.7% higher fees<br>**Fee Optimization Opportunities:**<br>1. Implement category-specific pricing tiers<br>2. Bundle small transactions (<$50) into larger settlements<br>3. Negotiate


key: chroma:document
string_value:
LU) show<br>0.5-0.7% higher fees<br>**Fee Optimization Opportunities:**<br>1. Implement category-specific pricing tiers<br>2. Bundle small transactions (<$50) into larger settlements<br>3. Negotiate lower rates for premium card transactions<br>4. Develop domestic processing alternatives for cross-border transactions<br>#### 3. Fraud Pattern Analysis<br>**High-risk factors:**<br>- **Digital goods** show 4.2% fraud rate (4.3× overall average)<br>- **Cross-border transactions** have 1.42% higher fraud incidence<br>- **Larger transactions** (>$500) show 2.1% fraud rate vs 0.8% for <$100<br>- **Card types:** Virtual/prepaid cards show 1.8× higher fraud rates<br>- **CVV absence:** Transactions without CVV have 3.2× higher fraud risk<br>**Fraud Reduction Strategies:**<br>1. Implement enhanced verification for digital goods merchants<br>2. Apply transaction limits and velocity checks for cross-border payments<br>3. Require CVV verification for transactions >$200<br>4. Develop ML-based fraud


key: chroma:document
string_value:
for digital goods merchants<br>2. Apply transaction limits and velocity checks for cross-border payments<br>3. Require CVV verification for transactions >$200<br>4. Develop ML-based fraud scoring models targeting high-risk categories<br>#### 4. Acquirer Selection Analysis<br>**Cost-Risk Tradeoffs by Location:**<br>| Acquirer Country | Avg Fee | Fraud Rate | Volume |<br>|------------------|---------|------------|--------|<br>| Germany (DE) | 2.05% | 0.82% | 32% |<br>| France (FR) | 2.12% | 0.91% | 28% |<br>| Cyprus (CY) | 2.63% | 1.24% | 8% |<br>| Luxembourg (LU) | 2.58% | 1.31% | 6% |<br>**Acquirer Optimization Strategies:**<br>1. Shift volume from high-cost acquirers (CY/LU) to DE/FR where possible<br>2. Implement multi-acquirer routing based on transaction profile<br>3. Negotiate volume-based discounts with primary acquirers<br>4. Develop failover systems to maintain uptime during routing changes<br>#### 5. Merchant-Specific Recommendations<br>**Digital Goods Merchants (High Fee,


key: chroma:document
string_value:
discounts with primary acquirers<br>4. Develop failover systems to maintain uptime during routing changes<br>#### 5. Merchant-Specific Recommendations<br>**Digital Goods Merchants (High Fee, High Fraud):**<br>- Implement 3D Secure for all transactions<br>- Negotiate flat-rate pricing instead of percentage-based<br>- Introduce micro-transaction pricing tiers<br>- Deploy behavioral biometrics for fraud prevention<br>**Retail Merchants (Medium Fee, Low Fraud):**<br>- Implement transaction size-based fee optimization<br>- Encourage standard card usage through loyalty programs<br>- Batch small transactions for processing efficiency<br>- Adopt address verification for high-value items<br>**Service Providers (Low Fee, Medium Fraud):**<br>- Implement recurring billing optimization<br>- Develop CVV-required policy for new customers<br>- Introduce cross-border fee disclosures<br>- Use predictive analytics for chargeback prevention<br>#### 6. Implementation Roadmap<br>1. **Phase 1 (0-3


key: chroma:document
string_value:
CVV-required policy for new customers<br>- Introduce cross-border fee disclosures<br>- Use predictive analytics for chargeback prevention<br>#### 6. Implementation Roadmap<br>1. **Phase 1 (0-3 months):** Implement high-impact fee optimizations (acquirer routing,<br>category-based pricing)<br>2. **Phase 2 (3-6 months):** Deploy fraud reduction systems (CVV enforcement, ML scoring)<br>3. **Phase 3 (6-12 months):** Develop advanced optimization (dynamic pricing, behavioral<br>biometrics)<br>**Expected Results:**<br>- 18-22% reduction in payment processing costs ($4.2-5.1M annually)<br>- 25-30% reduction in fraud losses ($1.8-2.2M annually)<br>- 1.5-2% improvement in conversion rates through optimized checkout flows<br>- Enhanced merchant satisfaction through tailored payment solutions<br>This comprehensive analysis provides actionable pathways to optimize payment processing<br>economics while maintaining security and conversion performance. Implementation should be<br>prioritized based


key: chroma:document
string_value:
comprehensive analysis provides actionable pathways to optimize payment processing<br>economics while maintaining security and conversion performance. Implementation should be<br>prioritized based on merchant-specific profiles and transaction patterns.|Below is the comprehensive report generated by our analysis:<br>------------------------------------------------------------<br>Data Science Report on Payment Transaction Dataset Analysis<br>------------------------------------------------------------<br>Overview:<br>This analysis examined fee structures, fraud patterns, and transaction characteristics from the<br>payment transaction dataset to identify strategies for cost optimization while preserving security<br>and conversion rates. Multiple data sources—fee details (fees.json), acquirer information<br>(acquirer_countries.csv), merchant profiles (merchant_data.json), merchant category codes<br>(merchant_category_codes.csv), and transaction records (payments.csv)—were integrated for


key: chroma:document
string_value:
information<br>(acquirer_countries.csv), merchant profiles (merchant_data.json), merchant category codes<br>(merchant_category_codes.csv), and transaction records (payments.csv)—were integrated for a<br>robust view.<br>Key Findings:<br>-------------<br>1. Fee Structure Drivers:<br>- Summary statistics indicate that fees correlate strongly with transaction amounts. High-value<br>transactions incur higher absolute fees, despite relatively stable fee percentages. This pattern<br>suggests that the fee structure is largely volume-driven.<br>- Fees.json data reveal that some acquirers apply additional fixed charges, exposing opportunities<br>for negotiations or alternative pricing models.<br>2. Fraud Pattern Analysis:<br>- Transactions flagged as fraudulent exhibit higher fee volatility and, occasionally, an elevated fee-<br>to-amount ratio. This may stem from extra chargeback or remediation costs.<br>- By isolating transactions with fraud flags, merchants can explore risk-based pricing


key: chroma:document
string_value:
occasionally, an elevated fee-<br>to-amount ratio. This may stem from extra chargeback or remediation costs.<br>- By isolating transactions with fraud flags, merchants can explore risk-based pricing strategies that<br>weigh fraud prevention costs against processing efficiency.<br>3. Acquirer Selection Strategies:<br>- Analysis of fee performance by acquirer shows significant variability. Certain acquirers,<br>especially those active in lower-risk regions, offer more competitive fee structures.<br>- When acquirer data is merged with country-level information, geographical differences in fee<br>formulations become apparent. This analysis suggests that merchants should strategically select<br>acquirers based on their target market’s risk profile and fee competitiveness.<br>4. Merchant Category and Profile Insights:<br>- Merging merchant transaction data with merchant category codes indicates that some merchant<br>categories (e.g., high-ticket retail or service sectors) face


key: chroma:document
string_value:
Category and Profile Insights:<br>- Merging merchant transaction data with merchant category codes indicates that some merchant<br>categories (e.g., high-ticket retail or service sectors) face disproportionately high fees.<br>- Merchant profile variables such as business size, transaction volume, and risk ratings (obtained<br>from merchant_data.json) greatly influence fee structures and fraud vulnerability. Tailored<br>strategies are thus needed depending on the merchant profile.<br>Actionable Recommendations:<br>-----------------------------<br>1. Optimizing Fee Structures:<br>- Negotiate with acquirers to reduce fixed fees, particularly for high-volume transactions. Consider<br>shifting to pricing models that reward transaction efficiency.<br>- Explore dynamic fee assessments where lower-risk, high-value transactions benefit from a<br>reduced per-transaction fee.<br>2. Fraud Risk Mitigation:<br>- Enhance fraud detection methods, especially for segments identified as high risk,


key: chroma:document
string_value:
lower-risk, high-value transactions benefit from a<br>reduced per-transaction fee.<br>2. Fraud Risk Mitigation:<br>- Enhance fraud detection methods, especially for segments identified as high risk, potentially<br>using machine learning-based anomaly detection before transactions are processed.<br>- Implement risk-based fee adjustments that incorporate security costs for transactions with higher<br>fraud potential.<br>3. Tailored Acquirer Selection:<br>- For merchants with international operations, perform data-driven assessments to select acquirers<br>with optimal fee structures within target markets.<br>- Continuously monitor acquirer performance (fee levels, fraud management, dispute resolution)<br>to ensure ongoing cost effectiveness.<br>4. Merchant-Specific Strategies:<br>- Small and medium-sized merchants should prioritize robust fraud prevention tools to reduce<br>exposure while controlling costs.<br>- High-volume merchants can leverage their scale for negotiating volume


key: chroma:document
string_value:
and medium-sized merchants should prioritize robust fraud prevention tools to reduce<br>exposure while controlling costs.<br>- High-volume merchants can leverage their scale for negotiating volume discounts or dedicated<br>agreements that improve fee structures.<br>Conclusion:<br>-----------<br>The analysis indicates that careful management of fee structures and acquirer selection—<br>combined with proactive fraud detection—can result in significant cost savings without<br>compromising security or conversion rates. Tailoring these recommendations based on merchant-<br>specific and regional nuances offers practical pathways for reducing overall payment processing<br>costs.<br>Next Steps:<br>-----------<br>- Further investigate temporal trends to refine dynamic pricing strategies.<br>- Invest in machine learning models for real-time fraud prediction and transaction scoring.<br>- Regularly re-assess acquirer agreements and fee strategies in light of evolving transaction data.<br>By


key: chroma:document
string_value:
Invest in machine learning models for real-time fraud prediction and transaction scoring.<br>- Regularly re-assess acquirer agreements and fee strategies in light of evolving transaction data.<br>By implementing these recommendations, merchants can optimize payment processing fees while<br>maintaining robust security and enhancing conversion outcomes.|


key: chroma:document
string_value:
_Figure 12._ A data analysis case of autonomous data science, from data sources to analyst-grade research reports. 

--- end of page.page_number=20 ---


key: chroma:document
string_value:
|Analyze the provided payments<br>dataset and supporting<br>documentation to produce a<br>comprehensive data insight report.<br>The report should identify the<br>primary drivers of transaction costs<br>and fraud risk for the merchants.<br>Investigate how factors such as<br>transaction characteristics (e.g.,<br>ACI, card scheme, amount),<br>merchant profiles (e.g., account<br>type, MCC, acquirer choice), and<br>behavioral patterns (e.g., time of<br>day, cross-border activity) influence<br>processing fees and fraudulent<br>disputes. The goal is to derive<br>actionable insights that can help<br>merchants optimize their payment<br>configurations and operational<br>practices to minimize costs and<br>mitigate risk.|# Comprehensive Analysis of Payment Processing Fees and Fraud Risk Factors<br>## Executive Summary<br>This comprehensive analysis examines the intricate relationship between payment transaction<br>characteristics, merchant profiles, and resulting processing fees and fraud risk.


key: chroma:document
string_value:
Executive Summary<br>This comprehensive analysis examines the intricate relationship between payment transaction<br>characteristics, merchant profiles, and resulting processing fees and fraud risk. Through systematic<br>analysis of over 23 million payment transactions across 30 merchant accounts, we identify key<br>drivers of transaction costs and fraud risk. The study reveals that card scheme selection accounts<br>for 68% of fee variation, while merchant category codes influence 42% of fraud risk profiles. Our<br>findings demonstrate that strategic configuration of payment processing parameters can reduce<br>costs by 18-22% while simultaneously decreasing fraudulent dispute rates by 14-17%.<br>## Methodology and Data Overview<br>The analysis leverages a comprehensive dataset comprising 23,028,700 payment transactions<br>processed through multiple acquirer networks between January 2022 and December 2023. The<br>dataset includes detailed transaction-level attributes including card


key: chroma:document
string_value:
23,028,700 payment transactions<br>processed through multiple acquirer networks between January 2022 and December 2023. The<br>dataset includes detailed transaction-level attributes including card scheme (Visa, Mastercard,<br>Amex, etc.), authorization characteristics (ACI codes), merchant category codes, and transaction<br>timing patterns. Merchant profiles include account types (Platform - Franchise, Retail, etc.),<br>capture delay settings, and historical fraud patterns. Fee structures were extracted from 1,000<br>distinct fee rules governing processing costs.<br>Statistical analysis employed multivariate regression modeling, decision tree analysis for fraud<br>pattern identification, and cost-benefit optimization modeling. Machine learning techniques<br>including random forest classification achieved 89.3% accuracy in predicting fraudulent<br>transactions when incorporating both transaction attributes and merchant profiles. The analysis<br>controlled for seasonal variations,


key: chroma:document
string_value:
achieved 89.3% accuracy in predicting fraudulent<br>transactions when incorporating both transaction attributes and merchant profiles. The analysis<br>controlled for seasonal variations, economic conditions, and geographic factors affecting payment<br>patterns.<br>## Key Findings on Cost Drivers<br>### Card Scheme and Authorization Characteristics<br>Card scheme selection emerges as the most significant determinant of processing costs,<br>accounting for 68% of fee variation according to our regression analysis. Visa transactions<br>averaged €0.1154 in fixed fees compared to €0.0833 for debit card transactions (p<0.001).<br>Authorization Characteristics Indicators (ACI) showed particularly strong cost implications, with<br>ACI 'B' transactions (Card Not Present - 3-D Secure) incurring 23% higher fees than ACI 'A'<br>(Card Present) transactions. The table below illustrates the cost differential by authorization type:<br>| ACI Code | Description | Fixed Fee (€) | Variable Rate (bps)


key: chroma:document
string_value:
23% higher fees than ACI 'A'<br>(Card Present) transactions. The table below illustrates the cost differential by authorization type:<br>| ACI Code | Description | Fixed Fee (€) | Variable Rate (bps) |<br>|----------|-------------|-----------------|---------------------|<br>| A | Card Present | 0.092 | 38.2 |<br>| B | 3-D Secure | 0.115 | 42.7 |<br>| C | Tokenized | 0.103 | 40.1 |<br>| D | Card On File | 0.121 | 45.3 |<br>| E | Recurring | 0.087 | 36.8 |<br>Merchant category codes demonstrated significant impact on both costs and fraud risk. Retail<br>merchants (MCC 5812) experienced 28% higher fraud rates than hospitality merchants (MCC<br>7999), while wholesale distributors (MCC 7372) enjoyed 19% lower dispute rates. Account type<br>proved equally impactful, with Platform Franchise accounts incurring 12% lower fees than Retail<br>accounts for equivalent transactions. The analysis identified three optimal fee optimization<br>opportunities yielding 18-22% cost savings through


key: chroma:document
string_value:
accounts incurring 12% lower fees than Retail<br>accounts for equivalent transactions. The analysis identified three optimal fee optimization<br>opportunities yielding 18-22% cost savings through strategic configuration of payment<br>parameters.<br>## Fraud Risk Patterns and Mitigation Strategies<br>Fraud risk patterns revealed distinct behavioral signatures across merchant segments. Cross-border<br>transactions (>15% of volume from non-primary markets) showed 37% higher fraud rates, while<br>transactions occurring outside normal business hours demonstrated 29% elevated risk. Machine<br>learning models identified four key fraud indicators with predictive accuracy exceeding 85%:<br>1. Device fingerprint mismatch (92% predictive power)<br>2. IP country mismatch (87% predictive power)<br>3. Transaction velocity anomalies (83% predictive power)<br>4. Historical fraud profile deviation (81% predictive power)<br>Merchant-specific risk profiles varied significantly by account type and


key: chroma:document
string_value:
Transaction velocity anomalies (83% predictive power)<br>4. Historical fraud profile deviation (81% predictive power)<br>Merchant-specific risk profiles varied significantly by account type and category. Platform<br>Franchise accounts demonstrated 19% lower fraud rates than Retail accounts for equivalent<br>transaction profiles, while Hospitality merchants showed 23% better fraud prevention outcomes<br>than Retail merchants. The analysis identified three optimal fraud prevention strategies yielding<br>14-17% reduction in fraudulent dispute rates through behavioral pattern optimization.<br>## Strategic Recommendations<br>### Cost Optimization Opportunities<br>Strategic configuration of payment parameters can yield significant cost savings while<br>maintaining service quality. Our analysis identifies three primary optimization opportunities:<br>1. Authorization Strategy Optimization:<br>- Shift 30% of ACI 'D' (Card On File) transactions to ACI 'E' (Recurring) for 12-15%


key: chroma:document
string_value:
Our analysis identifies three primary optimization opportunities:<br>1. Authorization Strategy Optimization:<br>- Shift 30% of ACI 'D' (Card On File) transactions to ACI 'E' (Recurring) for 12-15% cost<br>reduction<br>- Implement tokenization for 60% of card-not-present transactions to reduce fees by 8-10%<br>- Consolidate card schemes to two optimal options per merchant profile<br>2. Merchant Configuration Optimization:<br>- Align capture delay settings with business cycles (18% cost reduction)<br>- Optimize acquirer bank selection based on volume tiers (12-15% savings)<br>- Implement volume-based pricing tiers (9-11% savings)<br>3. Fraud Prevention Optimization:<br>- Implement behavioral pattern monitoring (14-17% fraud reduction)<br>- Optimize fraud rules by merchant segment (12-15% reduction)<br>- Implement dynamic fraud thresholds (10-13% reduction)<br>The comprehensive cost-benefit analysis demonstrates that implementation of these strategies can<br>yield 22-25% reduction in


key: chroma:document
string_value:
reduction)<br>- Implement dynamic fraud thresholds (10-13% reduction)<br>The comprehensive cost-benefit analysis demonstrates that implementation of these strategies can<br>yield 22-25% reduction in processing costs while simultaneously reducing fraudulent dispute rates<br>by 17-20%. The analysis further identifies specific configuration recommendations for each<br>merchant segment based on account type, volume profile, and risk tolerance.<br>## Conclusion and Future Research Directions<br>This comprehensive analysis demonstrates the significant impact of strategic payment processing<br>configuration on both cost efficiency and risk management. The findings challenge conventional<br>wisdom regarding fixed fee structures, demonstrating that dynamic, data-driven configuration of<br>payment parameters can yield substantial improvements in both financial and risk outcomes.<br>Future research should explore the integration of emerging technologies including AI-driven<br>authorization


key: chroma:document
string_value:
parameters can yield substantial improvements in both financial and risk outcomes.<br>Future research should explore the integration of emerging technologies including AI-driven<br>authorization decisioning and real-time fraud prevention systems. The demonstrated<br>methodologies provide a framework for continuous optimization of payment processing strategies<br>in an increasingly complex and competitive marketplace.|Based on the analysis performed, here are the key insights into transaction cost drivers and fraud<br>risk factors:<br>### 1. Primary Transaction Cost Drivers<br>- **Card Scheme Impact**: Visa transactions show significantly higher average fees (€X.XX)<br>compared to Mastercard (€X.XX)<br>- **Cross-Border Premium**: Cross-border transactions incur XX% higher fees on average<br>compared to domestic transactions<br>- **Account Type Differences**: Premium merchant accounts show XX% lower fee percentages<br>than standard accounts<br>- **ACI Influence**: High ACI scores


key: chroma:document
string_value:
to domestic transactions<br>- **Account Type Differences**: Premium merchant accounts show XX% lower fee percentages<br>than standard accounts<br>- **ACI Influence**: High ACI scores correlate with XX% lower fees due to reduced risk<br>surcharges<br>### 2. Fraud Risk Patterns<br>- **Temporal Patterns**: Highest dispute rates occur between [XX:XX-XX:XX] local time<br>- **Card Brand Variation**: [Card Brand] shows XX% higher dispute rates than others<br>- **MCC Risk Profile**: MCC codes [X] and [Y] show XX% higher fraud rates than average<br>- **Cross-Border Risk**: Cross-border transactions have XX% higher dispute rates<br>### 3. Fee Optimization Opportunities<br>- **MCC-Scheme Combinations**: [Scheme] + [MCC Description] shows XX% lower fees than<br>alternatives<br>- **Acquirer Selection**: Transactions routed through [Acquirer] show XX% lower fees<br>- **Volume Discounts**: Merchants processing >€XX/month qualify for XX% fee reductions<br>### 4. Operational Recommendations<br>1.


key: chroma:document
string_value:
Transactions routed through [Acquirer] show XX% lower fees<br>- **Volume Discounts**: Merchants processing >€XX/month qualify for XX% fee reductions<br>### 4. Operational Recommendations<br>1. Implement dynamic routing to prioritize [Scheme] for MCC [X] transactions<br>2. Add fraud screening enhancements during high-risk hours [XX:XX-XX:XX]<br>3. Migrate high-volume merchants to premium accounts for fee savings<br>4. Implement geolocation checks to flag high-risk cross-border patterns<br>The analysis reveals significant optimization potential through strategic payment configuration<br>adjustments and targeted fraud controls. Further segmentation by merchant size and transaction<br>volume could yield additional insights.|Below is a summary of the insights derived from our exploratory data analysis of the payments<br>dataset and supporting documentation:<br>─────────────────────────────<br>1. Transaction Characteristics & Card Scheme<br>─────────────────────────────<br>• Our aggregation


key: chroma:document
string_value:
of the payments<br>dataset and supporting documentation:<br>─────────────────────────────<br>1. Transaction Characteristics & Card Scheme<br>─────────────────────────────<br>• Our aggregation by card scheme shows that different brands can exhibit distinct behaviors. For<br>instance, some card schemes are associated with higher average transaction amounts while others<br>show a higher incidence (fraud rate) of fraudulent disputes. This suggests that merchants<br>accepting cards from schemes with higher fraud rates may face increased processing fees due to<br>the additional risk.<br>• Recommendation: Merchants can investigate alternative card acceptance strategies (or enforce<br>stricter verification methods) for cards linked to historically higher fraud incidences; additionally,<br>pricing or insurance adjustments may be considered.<br>─────────────────────────────<br>2. Behavioral Patterns: Time of Day<br>─────────────────────────────<br>• Analysis by the hour of the day indicates


key: chroma:document
string_value:
or insurance adjustments may be considered.<br>─────────────────────────────<br>2. Behavioral Patterns: Time of Day<br>─────────────────────────────<br>• Analysis by the hour of the day indicates that fraud incidents are not uniformly distributed.<br>Certain time slots (hours with higher fraud rates) are linked either to lower volumes or less<br>predictability in transactions. Such time windows could be targeted for enhanced monitoring and<br>fraud prevention controls.<br>• Recommendation: Merchant operations can consider operating additional fraud screening<br>measures during peak fraud risk hours, optimizing staffing decisions or real-time risk analytics at<br>these times.<br>─────────────────────────────<br>3. Cross-Border Activity<br>─────────────────────────────<br>• When investigating the role of cross-border versus domestic transactions, the data reveals that<br>cross-border transactions tend to have different fee profiles and may encounter higher fraud rates.<br>This may be


key: chroma:document
string_value:
the role of cross-border versus domestic transactions, the data reveals that<br>cross-border transactions tend to have different fee profiles and may encounter higher fraud rates.<br>This may be due to additional banking and regulatory complexities or differences in consumer<br>verification processes across countries.<br>• Recommendation: Merchants with substantial cross-border activity should focus on localizing<br>fraud prevention mechanisms, possibly integrating additional identity verification tools tailored to<br>international transactions, and reassessing fee configurations offered by acquirers regarding<br>international payments.<br>─────────────────────────────<br>4. Merchant Profiles and Operational Practices<br>─────────────────────────────<br>• By merging merchant profiles (from merchant_data.json) with transactions data, we identified<br>that merchant account type influences both transaction volumes and fraud incidence. For<br>instance, certain account types (potentially


key: chroma:document
string_value:
merchant_data.json) with transactions data, we identified<br>that merchant account type influences both transaction volumes and fraud incidence. For<br>instance, certain account types (potentially tied to higher risk industry classifications or MCC<br>codes) exhibit higher fraud rates alongside varying average transaction amounts.<br>• Recommendation: Merchants should review their merchant account profiles and consider if<br>their current configurations, acquirer choices, and business practices (such as transaction volume<br>patterns) are aligned with their risk tolerance. Further collaboration with acquirers on fee<br>structures that account for these risk profiles might help optimize processing costs.<br>─────────────────────────────<br>5. Additional Considerations: Fee Structures<br>─────────────────────────────<br>• Although the direct linkage between fee configuration (from fees.json) and transaction<br>characteristics was not explicitly modeled due to integration challenges, the


key: chroma:document
string_value:
Although the direct linkage between fee configuration (from fees.json) and transaction<br>characteristics was not explicitly modeled due to integration challenges, the available fee data<br>suggests that fee structures can vary based on parameters such as card scheme, transaction<br>amount, and risk profile.<br>• Recommendation: A further detailed analysis that integrates fee configurations with live<br>transactional data can uncover more granular relationships. Merchants may leverage such insights<br>to negotiate better fee agreements or adjust internal practices (e.g., reducing cross-border fraud<br>through better verification) that, in turn, optimize their fee liabilities.<br>─────────────────────────────<br>Final Summary:<br>─────────────────────────────<br>The analysis indicates that transaction characteristics (e.g. card scheme and transaction amount),<br>behavioral patterns (e.g. time-of-day and cross-border activity), and merchant-specific factors<br>(account type, MCC) all


key: chroma:document
string_value:
transaction characteristics (e.g. card scheme and transaction amount),<br>behavioral patterns (e.g. time-of-day and cross-border activity), and merchant-specific factors<br>(account type, MCC) all contribute as primary drivers of processing fees and fraud risk.<br>Merchants are advised to:<br>• Review card scheme mixtures and target risk mitigation strategies for higher-risk cards.<br>• Enhance fraud monitoring during identified high-risk hours.<br>• Implement tighter fraud controls for cross-border transactions.<br>• Reassess merchant profiles and operational configurations to align with fee structures and risk<br>profiles.<br>These insights offer actionable directions for merchants aiming to optimize their payment<br>configurations while controlling costs and mitigating risks.<br>─────────────────────────────<br>This comprehensive view reinforces that an integrated approach, using both transaction and<br>merchant profile data alongside fee configurations, is critical for reducing


key: chroma:document
string_value:
comprehensive view reinforces that an integrated approach, using both transaction and<br>merchant profile data alongside fee configurations, is critical for reducing exposure to fraud and<br>managing processing costs effectively.|


key: chroma:document
string_value:
_Figure 13._ A data insight case of autonomous data science, from data sources to analyst-grade research reports. 

--- end of page.page_number=21 ---


key: chroma:document
string_value:
|Based on the provided data (manual.md,<br>payments.csv, merchant_data.json,<br>fees.json, acquirer_countries.csv,<br>merchant_category_codes.csv), generate a<br>comprehensive **Data Report** analyzing<br>the payment processing operations for the<br>merchants. The report must follow the<br>exact structure and content outline<br>specified below.<br># Strategic Recommendations Report:<br>Payment Processing Analysis<br>## 1. Executive Summary<br>- Provide a high-level overview of key<br>findings and most critical<br>recommendations.<br>- Summarize overall performance, cost<br>efficiency, and fraud risk levels.<br>## 2. Introduction<br>- State the purpose and scope of the<br>analysis.<br>- Briefly describe the merchants and time<br>period covered.<br>## 3. Data Sources and Quality<br>- List and describe all data sources used in<br>the analysis.<br>- Assess data completeness, accuracy, and<br>any limitations encountered.<br>## 4. Analytical Approach<br>- Explain the methodology used for


key: chroma:document
string_value:
and describe all data sources used in<br>the analysis.<br>- Assess data completeness, accuracy, and<br>any limitations encountered.<br>## 4. Analytical Approach<br>- Explain the methodology used for fee<br>calculation and cost analysis.<br>- Describe how fraud risk and processing<br>efficiency were evaluated.<br>- Outline the criteria for identifying<br>optimization opportunities.<br>## 5. Findings<br>### 5.1 Transaction Volume and Patterns<br>- Analyze transaction volumes by<br>merchant, time, and card scheme.<br>- Identify seasonal trends and peak<br>processing periods.<br>### 5.2 Fee Structure Analysis<br>- Calculate and compare processing costs<br>across different merchants.<br>- Identify the most expensive fee<br>components and their drivers.<br>- Analyze how merchant characteristics<br>affect fee rates.<br>### 5.3 Fraud and Risk Assessment<br>- Evaluate fraud rates and dispute patterns.<br>- Assess the effectiveness of current fraud<br>prevention measures.<br>- Identify


key: chroma:document
string_value:
fee rates.<br>### 5.3 Fraud and Risk Assessment<br>- Evaluate fraud rates and dispute patterns.<br>- Assess the effectiveness of current fraud<br>prevention measures.<br>- Identify high-risk transaction types and<br>merchants.<br>### 5.4 Processing Efficiency<br>- Analyze authorization rates and refusal<br>patterns.<br>- Evaluate acquirer selection and routing<br>efficiency.<br>- Assess capture delay practices and their<br>cost implications.<br>## 6. Limitations<br>- Discuss any data constraints or<br>methodological limitations.<br>- Identify areas where additional data<br>would improve analysis.<br>## 7. Strategic Recommendations<br>### 7.1 Cost Optimization Strategies<br>- Specific recommendations for reducing<br>processing fees.<br>- Suggestions for optimizing transaction<br>routing and acquirer selection.<br>### 7.2 Fraud Prevention Improvements<br>- Recommendations for enhancing security<br>measures.<br>- Strategies for reducing fraud-related costs<br>and chargebacks.<br>### 7.3


key: chroma:document
string_value:
acquirer selection.<br>### 7.2 Fraud Prevention Improvements<br>- Recommendations for enhancing security<br>measures.<br>- Strategies for reducing fraud-related costs<br>and chargebacks.<br>### 7.3 Operational Efficiency<br>Enhancements<br>- Suggestions for improving authorization<br>rates and reducing refusals.<br>- Recommendations for optimizing<br>capture timing and settlement processes.<br>## 8. Conclusion<br>- Summarize key insights and expected<br>benefits of implemented<br>recommendations.<br>- Outline next steps for implementation<br>and monitoring.<br>## 9. Bibliography<br>- List all referenced documents, manuals,<br>and data sources.<br>## 10. Appendix<br>- Include supporting tables, charts, and<br>detailed calculations.<br>- Provide merchant-specific analysis and<br>recommendations.<br>Ensure the report is data-driven, with<br>specific metrics and calculations<br>supporting all findings and<br>recommendations. Use clear, professional<br>language suitable for


key: chroma:document
string_value:
the report is data-driven, with<br>specific metrics and calculations<br>supporting all findings and<br>recommendations. Use clear, professional<br>language suitable for business<br>stakeholders.|# Strategic Recommendations Report: Payment Processing Analysis<br>## 1. Executive Summary<br>This comprehensive analysis of payment processing operations reveals significant opportunities for cost optimization and<br>operational improvement. The dataset encompasses 1,000 transactions across 30 merchants, with an average processing<br>fee of €0.6574 per transaction, comprising €0.8058 in fixed fees and €6.3354 in variable rate fees per €10,000 processed.<br>The analysis identifies three primary areas for strategic improvement: fee structure optimization (35% potential savings),<br>fraud prevention enhancement (22% reduction target), and operational efficiency gains (15% improvement). Merchant<br>segmentation reveals significant variation in fee structures, with cross-border transactions


key: chroma:document
string_value:
enhancement (22% reduction target), and operational efficiency gains (15% improvement). Merchant<br>segmentation reveals significant variation in fee structures, with cross-border transactions averaging 18% higher fees than<br>domestic transactions. The data demonstrates that 23% of transactions exhibit characteristics warranting enhanced fraud<br>controls, while 17% of merchants operate with suboptimal acquirer relationships. These findings enable targeted<br>interventions to improve profitability while maintaining service quality.<br>## 2. Introduction<br>This report analyzes payment processing operations for a network of 30 merchants across multiple European countries,<br>covering transaction data from 2023. The analysis incorporates transactional records, merchant attributes, fee structures,<br>and acquirer relationships to evaluate operational efficiency and identify optimization opportunities. The dataset includes<br>1,000 sampled transactions with an average value of €87.42,


key: chroma:document
string_value:
acquirer relationships to evaluate operational efficiency and identify optimization opportunities. The dataset includes<br>1,000 sampled transactions with an average value of €87.42, representing €87,420 in total processed volume. Merchant<br>categories span retail (42%), hospitality (23%), and professional services (35%), with merchant category codes following<br>ISO 8583 standards. The analysis period covers a complete calendar year, capturing seasonal patterns and business cycle<br>effects. Data quality assessment indicates 98.7% completeness in critical fields, with missing values concentrated in<br>optional fields like email addresses (1.3% missing). The methodology combines quantitative analysis of transaction<br>patterns with rule-based fee calculations against the 1,000 fee rules in the system.<br>## 3. Data Sources and Quality<br>The analysis integrates six primary data sources, each contributing unique dimensions to the payment processing<br>analysis. The payments dataset


key: chroma:document
string_value:
in the system.<br>## 3. Data Sources and Quality<br>The analysis integrates six primary data sources, each contributing unique dimensions to the payment processing<br>analysis. The payments dataset forms the transactional foundation, containing 21 fields capturing merchant, card<br>scheme, timing, and amount details. Merchant attributes from merchant_data.json provide 5 key attributes per merchant,<br>while fees.json contains 1,000 complex fee rules with 12 conditional fields each. Merchant category codes provide<br>standardized industry classifications, and acquirer countries map 8 payment processors to their respective countries. Data<br>quality assessment reveals excellent completeness (98.2% complete records) with minimal missing values concentrated in<br>optional fields. The fee rules demonstrate comprehensive coverage, with 92% of transactions matching at least one fee<br>rule and 78% matching exactly one rule. Validation against business rules in manual.md confirms structural


key: chroma:document
string_value:
demonstrate comprehensive coverage, with 92% of transactions matching at least one fee<br>rule and 78% matching exactly one rule. Validation against business rules in manual.md confirms structural compliance<br>with payment processing standards. Limitations include the 1,000-transaction sample size and lack of historical<br>comparisons, though the analysis accounts for these through statistical techniques.<br>## 4. Analytical Approach<br>The methodology combines descriptive analytics, predictive modeling, and prescriptive optimization to evaluate payment<br>processing operations. Fee calculations implement the formula: Fee = FixedAmount + (Rate× TransactionValue /<br>10,000), applying the first matching rule from 12 conditional dimensions. Fraud risk assessment combines transaction<br>velocity, geographic patterns, and historical fraud rates using a logistic regression model with AUC-ROC of 0.82.<br>Processing efficiency metrics include authorization rates (87.3%), refusal rates


key: chroma:document
string_value:
geographic patterns, and historical fraud rates using a logistic regression model with AUC-ROC of 0.82.<br>Processing efficiency metrics include authorization rates (87.3%), refusal rates (9.1%), and capture delay patterns.<br>Optimization models employ linear programming to identify acquirer routing improvements, achieving 14% simulated<br>savings. Merchant segmentation uses k-means clustering (k=5) on transaction patterns and fee structures. All analyses<br>implement 5-fold cross-validation to ensure robustness, with key metrics showing <5% variation across folds. The<br>approach balances statistical rigor with business interpretability, ensuring actionable insights.<br>## 5. Findings<br>### 5.1 Transaction Volume and Patterns<br>Transaction analysis reveals distinct patterns across merchants, card schemes, and time periods. Volume peaks between<br>11am-2pm (28% of daily volume) and 6-9pm (31%), with weekday volumes 42% higher than weekends. Cross-border<br>transactions (issuing


key: chroma:document
string_value:
merchants, card schemes, and time periods. Volume peaks between<br>11am-2pm (28% of daily volume) and 6-9pm (31%), with weekday volumes 42% higher than weekends. Cross-border<br>transactions (issuing country ≠ acquirer country) represent 38% of volume but 53% of fees, averaging €0.72/transaction<br>versus €0.61 for domestic. Card scheme analysis shows GlobalCard processing 22% more transactions than competitors,<br>though with 18% higher fees. Merchant category analysis reveals retail merchants process 3.2 transactions/merchant<br>versus 1.8 for professional services, with average transaction values of €78.42 and €124.57 respectively. Fraud patterns<br>show 2.1% overall fraud rate, but 4.7% for high-value (>€500) credit card transactions. Seasonal analysis identifies<br>November peaks (18% above average) and February troughs (23% below average).<br>### 5.2 Fee Structure Analysis<br>Fee analysis reveals significant variation across merchants and conditions. The average fee structure


key: chroma:document
string_value:
above average) and February troughs (23% below average).<br>### 5.2 Fee Structure Analysis<br>Fee analysis reveals significant variation across merchants and conditions. The average fee structure contains 4.2<br>conditional dimensions per rule, with capture delay (32% of rules), merchant category (28%), and card scheme (22%)<br>being most common conditions. Fixed fees range from €0.05 to €0.25 (mean €0.12), while variable rates range from 15 to<br>95 basis points (mean 42 bps). Merchant segmentation identifies three distinct fee profiles: low-volume/high-margin (12<br>merchants), volume-driven (15 merchants), and cross-border focused (3 merchants). Fee optimization modeling identifies<br>potential savings of €1,200 annually through acquirer routing changes alone. Intracountry transactions enjoy 18% lower<br>fees on average, suggesting substantial routing optimization opportunities. The analysis identifies 7 merchants operating<br>with suboptimal acquirer relationships, representing


key: chroma:document
string_value:
enjoy 18% lower<br>fees on average, suggesting substantial routing optimization opportunities. The analysis identifies 7 merchants operating<br>with suboptimal acquirer relationships, representing €8,400 in potential savings.<br>### 5.3 Fraud and Risk Assessment<br>Fraud analysis identifies four high-risk merchant categories: travel (7.2% fraud rate), electronics (6.8%), retail (5.1%), and<br>hospitality (4.9%). Transaction velocity analysis flags 23% of transactions exceeding historical norms, while geographic<br>analysis identifies 17% of transactions with unusual country mismatches. Machine learning models predict fraud with<br>82% accuracy, identifying 14% of transactions requiring manual review. Capture delay analysis shows merchants with >7<br>day delays experience 38% higher chargebacks. Fraud prevention improvements could reduce losses by €1,200 annually<br>through enhanced velocity checks and 3D Secure implementation. The analysis identifies 8 merchants with


key: chroma:document
string_value:
38% higher chargebacks. Fraud prevention improvements could reduce losses by €1,200 annually<br>through enhanced velocity checks and 3D Secure implementation. The analysis identifies 8 merchants with suboptimal<br>fraud controls, representing €9,800 in potential savings.<br>### 5.4 Processing Efficiency<br>Authorization analysis reveals 87.3% authorization rates, with debit cards (92%) outperforming credit (82%). Refusal<br>analysis identifies acquirer routing as primary cause (63%), followed by card scheme restrictions (22%) and merchant<br>limits (15%). Capture delay analysis shows merchants with <3 day delays achieve 28% lower dispute rates. Settlement<br>timing analysis identifies 17% of merchants settling >T+5, correlating with 31% higher late fees. Routing optimization<br>modeling identifies €1,400 in potential savings through acquirer reassignment. Authorization rate improvements could<br>yield €2,100 annually through improved merchant training. The analysis identifies 12


key: chroma:document
string_value:
identifies €1,400 in potential savings through acquirer reassignment. Authorization rate improvements could<br>yield €2,100 annually through improved merchant training. The analysis identifies 12 merchants with suboptimal<br>processing practices, representing €15,200 in potential savings.<br>## 6. Limitations<br>The analysis faces several constraints requiring cautious interpretation. The 1,000-transaction sample, while<br>representative, may not capture extreme values adequately. The static fee rules don't account for dynamic pricing,<br>potentially underestimating elasticity. Fraud models lack real-time transaction data, potentially missing emerging<br>patterns. The merchant segmentation uses only 5 features, potentially oversimplifying complex business models. The<br>analysis period covers one calendar year, limiting seasonal comparisons. Data limitations include missing merchant bank<br>account details and limited historical benchmarking. The fee optimization models assume static


key: chroma:document
string_value:
one calendar year, limiting seasonal comparisons. Data limitations include missing merchant bank<br>account details and limited historical benchmarking. The fee optimization models assume static merchant volumes,<br>while actual volumes may fluctuate. These limitations are mitigated through conservative confidence intervals and<br>sensitivity analysis. Future research could incorporate real-time data streams and machine learning for dynamic pricing.<br>## 7. Strategic Recommendations<br>### 7.1 Cost Optimization Strategies<br>Fee optimization should prioritize three initiatives: acquirer routing (€1,400 potential), fee structure review (€1,200), and<br>capture delay reduction (€1,500). Merchant-specific analysis identifies 7 merchants with €8,400 in routing savings and 3<br>with €9,800 in fraud prevention improvements. The analysis recommends implementing dynamic routing algorithms to<br>capture 12% of potential savings automatically. Fee structure review should focus on cross-border


key: chroma:document
string_value:
fraud prevention improvements. The analysis recommends implementing dynamic routing algorithms to<br>capture 12% of potential savings automatically. Fee structure review should focus on cross-border transactions, where<br>fees average 18% higher. Capture delay improvements should target merchants with >7 day delays, potentially saving<br>€2,100 annually. The implementation roadmap prioritizes quick wins (routing) before complex changes (fee<br>restructuring). The projected 28% annual savings (€38,900) delivers 3:1 ROI within 6 months. The program should<br>include ongoing monitoring with monthly KPIs and quarterly reviews.<br>### 7.2 Fraud Prevention Improvements<br>Fraud prevention should prioritize three areas: velocity controls (€1,200 potential), 3D Secure implementation (€900),<br>and merchant training (€700). The analysis identifies 8 merchants with €9,800 in fraud prevention improvements.<br>Velocity controls should focus on high-value merchants (>$500 transactions), where


key: chroma:document
string_value:
merchant training (€700). The analysis identifies 8 merchants with €9,800 in fraud prevention improvements.<br>Velocity controls should focus on high-value merchants (>$500 transactions), where fraud rates peak at 4.7%. 3D Secure<br>implementation should target credit card transactions, where fraud rates peak at 5.1%. Merchant training should focus on<br>high-risk categories (travel, electronics). The implementation roadmap prioritizes quick wins (velocity controls) before<br>complex changes (3D Secure). The projected 22% reduction in fraud losses (€2,100) delivers 3:1 ROI within 9 months.<br>The program should include real-time monitoring with daily dashboards and weekly reviews.<br>### 7.3 Operational Efficiency Enhancements<br>Processing efficiency should prioritize three areas: authorization rates (€2,100 potential), settlement timing (€1,500), and<br>acquirer selection (€1,400). The analysis identifies 12 merchants with €15,200 in processing improvements.<br>Authorization rate


key: chroma:document
string_value:
rates (€2,100 potential), settlement timing (€1,500), and<br>acquirer selection (€1,400). The analysis identifies 12 merchants with €15,200 in processing improvements.<br>Authorization rate improvements should focus on credit card transactions, where rates lag debit by 10 percentage points.<br>Settlement timing should target merchants with >T+5 delays, where fees peak at 31% higher. Acquirer selection should<br>focus on cross-border transactions, where fees peak at 18% higher. The implementation roadmap prioritizes quick wins<br>(authorization training) before complex changes (acquirer reassignment). The projected 15% improvement (€22,700)<br>delivers 3:1 ROI within 12 months. The program should include real-time monitoring with daily dashboards and<br>monthly reviews.<br>## 8. Conclusion<br>This comprehensive analysis identifies substantial opportunities to improve payment processing operations through data-<br>driven optimization. The findings demonstrate that strategic


key: chroma:document
string_value:
Conclusion<br>This comprehensive analysis identifies substantial opportunities to improve payment processing operations through data-<br>driven optimization. The findings demonstrate that strategic interventions can yield €38,900 in annual savings (28% of<br>current costs) while improving service quality. The recommended initiatives combine quick wins with longer-term<br>transformations, delivering measurable improvements across cost, risk, and efficiency metrics. Implementation requires<br>cross-functional collaboration between operations, risk, and finance teams. The proposed roadmap balances immediate<br>action with sustainable transformation, ensuring lasting impact. The analysis provides the foundation for ongoing<br>performance management, with recommended metrics for continuous improvement. Future research should incorporate<br>real-time data streams and machine learning for dynamic optimization. The findings demonstrate the transformative<br>potential of data analytics in


key: chroma:document
string_value:
improvement. Future research should incorporate<br>real-time data streams and machine learning for dynamic optimization. The findings demonstrate the transformative<br>potential of data analytics in payment processing operations.<br>## 9. Bibliography<br>Merchant Guide to Optimizing Payment Processing and Minimizing Fees (v2.1, November 2024)<br>Payment Processing Standards (ISO 8583:2021)<br>Merchant Category Code Reference (v3.2, October 2024)<br>Acquirer Country Mapping (v1.5, September 2024)<br>Fee Rule Specifications (v2.0, August 2024)<br>Data Dictionary (v1.0, July 2024)<br>## 10. Appendix<br>Appendix A: Merchant Segmentation Details<br>Appendix B: Fee Rule Matching Algorithm<br>Appendix C: Fraud Model Specifications<br>Appendix D: Optimization Model Parameters<br>Appendix E: Implementation Roadmaps<br>Appendix F: Detailed Merchant Analysis<br>Appendix G: Technical Specifications|I'll analyze the data and prepare the comprehensive report based on the code execution results.


key: chroma:document
string_value:
Roadmaps<br>Appendix F: Detailed Merchant Analysis<br>Appendix G: Technical Specifications|I'll analyze the data and prepare the comprehensive report based on the code execution results. Here's the report:<br>### Strategic Recommendations Report: Payment Processing Analysis<br>#### 1. Executive Summary<br>- Overall transaction processing shows moderate efficiency with an authorization rate of 82.3%<br>- Fraud rate is concerning at 3.5%, significantly impacting revenue<br>- Dispute rate of 1.2% indicates potential customer satisfaction issues<br>- Highest costs come from interchange fees (average 0.45% per transaction)<br>- Capture delay shows room for improvement with average 1.8 hours between creation and capture<br>#### 2. Introduction<br>- Analysis covers payment processing for multiple merchants over an unspecified period<br>- Purpose: Identify cost optimization, fraud reduction, and efficiency improvements<br>- Scope: Transaction patterns, fee structures, fraud assessment, and


key: chroma:document
string_value:
merchants over an unspecified period<br>- Purpose: Identify cost optimization, fraud reduction, and efficiency improvements<br>- Scope: Transaction patterns, fee structures, fraud assessment, and processing efficiency<br>#### 3. Data Sources and Quality<br>- **payments.csv (23MB):** Core transaction data with authorization status, fraud flags, and timestamps<br>- **merchant_data.json (6.7KB):** Merchant profiles including IDs and categories<br>- **fees.json (518.7KB):** Detailed fee structures by merchant and transaction type<br>- **acquirer_countries.csv (0.2KB):** Acquirer country codes and names<br>- **merchant_category_codes.csv (26KB):** MCC category descriptions<br>- **Limitations:** Date range not specified in sample data; merchant names not included<br>#### 4. Analytical Approach<br>- Fee calculation: Summed interchange, scheme, and acquirer fee components<br>- Fraud risk: Calculated fraud and dispute rates; identified high-risk merchants<br>- Processing efficiency: Analyzed


key: chroma:document
string_value:
Fee calculation: Summed interchange, scheme, and acquirer fee components<br>- Fraud risk: Calculated fraud and dispute rates; identified high-risk merchants<br>- Processing efficiency: Analyzed authorization rates, refusal reasons, and capture delays<br>- Optimization criteria: Cost reduction potential, fraud mitigation impact, and efficiency gains<br>#### 5. Findings<br>##### 5.1 Transaction Volume and Patterns<br>- Transaction distribution:<br>- Visa: 38% of transactions<br>- Mastercard: 35%<br>- American Express: 27%<br>- Strong seasonal patterns: 15% higher volume in December<br>- Daily peak: 3-4 PM (18% of daily transactions)<br>##### 5.2 Fee Structure Analysis<br>- Most expensive components:<br>1. Cross-border interchange fee (0.32%)<br>2. Scheme fee (0.25%)<br>3. Acquirer markup (0.18%)<br>- Fee variations by merchant size: Smaller merchants pay 22% higher fees on average<br>- High-MCC merchants (hotels, car rentals) incur 30% higher fees<br>##### 5.3 Fraud and Risk


key: chroma:document
string_value:
markup (0.18%)<br>- Fee variations by merchant size: Smaller merchants pay 22% higher fees on average<br>- High-MCC merchants (hotels, car rentals) incur 30% higher fees<br>##### 5.3 Fraud and Risk Assessment<br>- Highest fraud rates by merchant:<br>- Merchant X: 8.2%<br>- Merchant Y: 6.7%<br>- Merchant Z: 5.9%<br>- Card scheme fraud rates:<br>- American Express: 4.8%<br>- Visa: 3.1%<br>- Mastercard: 2.9%<br>##### 5.4 Processing Efficiency<br>- Authorization rate: 82.3%<br>- Top refusal reasons:<br>- Insufficient funds: 45%<br>- Suspected fraud: 30%<br>- Invalid card: 15%<br>- Capture delay statistics:<br>- Mean: 1.8 hours<br>- Max: 48 hours<br>- 90th percentile: 5 hours<br>#### 6. Limitations<br>- No transaction amount data for precise revenue impact calculation<br>- Limited merchant details prevent granular category analysis<br>- Time period not specified limits trend analysis<br>- Missing chargeback data for complete fraud assessment<br>#### 7. Strategic Recommendations<br>#####


key: chroma:document
string_value:
prevent granular category analysis<br>- Time period not specified limits trend analysis<br>- Missing chargeback data for complete fraud assessment<br>#### 7. Strategic Recommendations<br>##### 7.1 Cost Optimization Strategies<br>- Negotiate interchange fees for high-volume merchants (potential 15% savings)<br>- Implement MCC optimization for high-cost categories<br>- Consolidate acquirer relationships to reduce markup fees<br>##### 7.2 Fraud Prevention Improvements<br>- Introduce machine learning fraud scoring for high-risk merchants<br>- Implement 3D Secure for American Express transactions<br>- Create merchant-specific fraud thresholds based on historical patterns<br>##### 7.3 Operational Efficiency Enhancements<br>- Optimize transaction routing to acquirers with >90% authorization rates<br>- Implement automated retry system for \"insufficient funds\" declines<br>- Reduce capture delay target to 1 hour (potential 0.2% fee reduction)<br>#### 8. Conclusion<br>- Implementing


key: chroma:document
string_value:
rates<br>- Implement automated retry system for \"insufficient funds\" declines<br>- Reduce capture delay target to 1 hour (potential 0.2% fee reduction)<br>#### 8. Conclusion<br>- Implementing recommendations could reduce processing costs by 18-22%<br>- Fraud reduction initiatives could save $500K+ annually for mid-sized merchants<br>- Efficiency improvements could increase revenue capture by 5-7%<br>- Next steps: Prioritize cost optimization and fraud prevention for highest-impact merchants<br>#### 9. Bibliography<br>1. Payment Processing Manual (manual.md)<br>2. Transaction Data (payments.csv)<br>3. Merchant Profiles (merchant_data.json)<br>4. Fee Schedule (fees.json)<br>5. Acquirer Reference (acquirer_countries.csv)<br>6. MCC Reference (merchant_category_codes.csv)<br>#### 10. Appendix<br>- Detailed fee analysis by merchant available upon request<br>- Fraud pattern charts by card scheme and time of day<br>- Capture delay distribution histogram<br>- Acquirer performance comparison


key: chroma:document
string_value:
Detailed fee analysis by merchant available upon request<br>- Fraud pattern charts by card scheme and time of day<br>- Capture delay distribution histogram<br>- Acquirer performance comparison table<br>This report provides data-driven insights to optimize payment operations, reduce costs, and mitigate fraud risks across<br>your merchant portfolio.|Below is the comprehensive Data Report generated based on the analysis of the provided payment processing data:<br>Strategic Recommendations Report: Payment Processing Analysis<br>1. Executive Summary<br>------------------------------------------------------------------<br>- Key Findings:<br>- Total Transactions Processed: [Extracted from payments.csv]<br>- Merchants Covered: [Extracted from payments.csv]<br>- Significant variation noted in fee components and processing times.<br>- Recommendations:<br>- Optimize acquirer selection and fee structures.<br>- Enhance fraud detection measures for high-risk transaction types.<br>2.


key: chroma:document
string_value:
noted in fee components and processing times.<br>- Recommendations:<br>- Optimize acquirer selection and fee structures.<br>- Enhance fraud detection measures for high-risk transaction types.<br>2. Introduction<br>------------------------------------------------------------------<br>- Purpose: To analyze payment processing operations with an emphasis on cost efficiency, risk, and operational<br>performance.<br>- Scope: The analysis covers multiple merchants over the time period as indicated in the transaction data.<br>3. Data Sources and Quality<br>------------------------------------------------------------------<br>- Data Sources:<br>- manual.md: Operational procedures and system guidelines.<br>- payments.csv: Transaction records with detailed payment information.<br>- merchant_data.json: Profiles and characteristics of merchants.<br>- fees.json: Detailed fee structures per transaction and merchant.<br>- acquirer_countries.csv: Geographical distribution of acquirers.<br>-


key: chroma:document
string_value:
Profiles and characteristics of merchants.<br>- fees.json: Detailed fee structures per transaction and merchant.<br>- acquirer_countries.csv: Geographical distribution of acquirers.<br>- merchant_category_codes.csv: Classification of merchants by business types.<br>- Data Quality Assessment:<br>- Payments data is extensive but has some missing fields (e.g., transaction dates) that might impact temporal analyses.<br>- Fees data provides detailed breakdowns; however, its complexity requires careful reconciliation with merchant profiles.<br>- Overall, data is adequate for in-depth analysis, though supplementary granular data (especially on fraud incidents)<br>would further enhance insights.<br>4. Analytical Approach<br>------------------------------------------------------------------<br>- Methodology:<br>- Fee Calculation: Aggregated fees per merchant were compared to reveal variations in cost efficiency. Major fee drivers<br>included processing fees, acquirer service charges, and


key: chroma:document
string_value:
Methodology:<br>- Fee Calculation: Aggregated fees per merchant were compared to reveal variations in cost efficiency. Major fee drivers<br>included processing fees, acquirer service charges, and additional service fees.<br>- Fraud Risk Evaluation: Analyzed fraud indicators via dispute records in the payments data correlated against merchant<br>risk profiles from merchant_data.json.<br>- Processing Efficiency: Evaluated key metrics like authorization and capture rates, refusal patterns, and acquirer routing<br>efficiency using the payments.csv data.<br>- Optimization Criteria: Identified high fee components, elevated fraud signals, and sub-optimal processing speeds as<br>primary opportunities for operational improvements.<br>5. Findings<br>------------------------------------------------------------------<br>5.1 Transaction Volume and Patterns<br>- Total transactions processed: Data from payments.csv shows a large volume across the analyzed period.<br>- Seasonal trends: Evidence of


key: chroma:document
string_value:
Transaction Volume and Patterns<br>- Total transactions processed: Data from payments.csv shows a large volume across the analyzed period.<br>- Seasonal trends: Evidence of peaks during promotional periods and seasonal events.<br>- Card scheme insights: Diverse usage across major card networks (Visa, MasterCard, etc.) indicate varied consumer<br>behaviors.<br>5.2 Fee Structure Analysis<br>- Variability in fee structures: Fees vary notably across merchants, influenced by operational volume and merchant<br>category.<br>- Major fee components: Processing fees and acquirer service charges represent the highest cost elements.<br>- Impact of merchant characteristics: Analysis suggests that merchant category (as indexed in<br>merchant_category_codes.csv) and volume are key determinants driving fee rates.<br>5.3 Fraud and Risk Assessment<br>- Fraud trends: Some segments, particularly those with high transaction volumes, show elevated dispute rates.<br>- Effectiveness of fraud prevention:


key: chroma:document
string_value:
fee rates.<br>5.3 Fraud and Risk Assessment<br>- Fraud trends: Some segments, particularly those with high transaction volumes, show elevated dispute rates.<br>- Effectiveness of fraud prevention: Current measures are moderately effective; however, enhanced real-time monitoring<br>could reduce fraud-related costs.<br>- High-risk identification: Certain transaction types and specific merchants are flagged for heightened fraud risk.<br>5.4 Processing Efficiency<br>- Authorization vs. refusal: Most merchants experience industry-standard authorization rates, though a subset shows<br>higher refusal numbers.<br>- Acquirer selection: Analysis based on acquirer_countries.csv reveals potential routing improvements that could lower<br>fees.<br>- Capture delays: Delay in capture processes directly adds to cost overheads, suggesting a need for more timely settlement<br>processes.<br>6. Limitations<br>------------------------------------------------------------------<br>- Data Constraints: Some


key: chroma:document
string_value:
adds to cost overheads, suggesting a need for more timely settlement<br>processes.<br>6. Limitations<br>------------------------------------------------------------------<br>- Data Constraints: Some fields in payments.csv (such as transaction dates) were incomplete or inconsistently formatted,<br>affecting the precision of time-based analyses.<br>- Methodological Limitations: Assumptions made during fee aggregation might not fully capture all fee nuances.<br>- Additional Data: Detailed data on fraud incidents and more granular merchant performance metrics would provide<br>enhanced clarity.<br>7. Strategic Recommendations<br>------------------------------------------------------------------<br>7.1 Cost Optimization Strategies<br>- Negotiate fee rates by leveraging merchant’s high-volume statistics to secure better terms.<br>- Re-evaluate acquirer and transaction routing decisions to identify efficient cost reductions based on regional<br>performance data from


key: chroma:document
string_value:
merchant’s high-volume statistics to secure better terms.<br>- Re-evaluate acquirer and transaction routing decisions to identify efficient cost reductions based on regional<br>performance data from acquirer_countries.csv.<br>7.2 Fraud Prevention Improvements<br>- Enhance real-time fraud detection systems and update risk models using detailed merchant data.<br>- Implement targeted monitoring for high-risk transaction segments identified in the analysis.<br>7.3 Operational Efficiency Enhancements<br>- Streamline authorization processes by identifying and mitigating common refusal reasons.<br>- Reduce capture delays through improved system integration and proactive settlement workflows.<br>8. Conclusion<br>------------------------------------------------------------------<br>- Key Insights: The analysis underscores the need for improvements in reducing processing fees, strengthening fraud<br>prevention, and enhancing overall operational efficiency.<br>- Next Steps: Implement the


key: chroma:document
string_value:
The analysis underscores the need for improvements in reducing processing fees, strengthening fraud<br>prevention, and enhancing overall operational efficiency.<br>- Next Steps: Implement the recommended optimizations, closely monitor the outcomes with real-time dashboards, and<br>continually refine fraud risk models.<br>9. Bibliography<br>------------------------------------------------------------------<br>- manual.md<br>- payments-readme.md<br>- payments.csv<br>- merchant_data.json<br>- fees.json<br>- acquirer_countries.csv<br>- merchant_category_codes.csv<br>10. Appendix<br>------------------------------------------------------------------<br>- Detailed tables and visualizations (e.g., fee breakdowns, seasonal transaction trends, and risk assessments) underpin the<br>findings.<br>- Merchant-specific analysis and customized recommendations are available upon request.<br>This report is data-driven and built upon specific metrics and calculations derived from the provided data


key: chroma:document
string_value:
Merchant-specific analysis and customized recommendations are available upon request.<br>This report is data-driven and built upon specific metrics and calculations derived from the provided data sources. The<br>recommendations are aimed at reducing costs, mitigating fraud risk, and optimizing operational efficiency in the<br>payment processing environment.|


key: chroma:document
string_value:
_Figure 14._ A case of autonomous data science with report constraints. 

--- end of page.page_number=22 ---


key: chroma:document
string_value:
|Generate a report summarizing any<br>notable trends, findings, or<br>summaries in the data.|# Comprehensive Analysis of Payment Processing Data: Trends, Patterns, and Business Insights<br>## Executive Summary<br>This comprehensive analysis examines a rich dataset encompassing merchant payment<br>transactions, fee structures, and merchant characteristics. The study reveals significant insights<br>into payment processing patterns, fraud risks, and fee optimization opportunities. Through<br>multidimensional analysis of over 138,000 transactions across 30 merchants, we identify<br>actionable insights for payment processors and merchants alike. The findings demonstrate<br>substantial variation in fraud rates (ranging from 1.2% to 18.7% across merchant categories), fee<br>structures (varying from €0.01 to €0.14 per transaction), and transaction patterns (peak volumes at<br>12:00-13:00 hours).<br>## Merchant Category Analysis and Fraud Patterns<br>The merchant category analysis reveals


key: chroma:document
string_value:
from €0.01 to €0.14 per transaction), and transaction patterns (peak volumes at<br>12:00-13:00 hours).<br>## Merchant Category Analysis and Fraud Patterns<br>The merchant category analysis reveals significant variation in transaction volumes and fraud<br>rates across different business sectors. The top merchant category (code 7997) accounts for 28.7%<br>of transactions in our sample, followed by category 7993 (14.3%) and 7372 (14.1%). Fraud rates<br>show substantial variation across categories, with the highest fraud rates observed in categories<br>5812 (18.7%) and 5942 (16.9%), compared to the overall average of 6.3%. These findings suggest<br>that certain merchant categories require enhanced fraud prevention measures and potentially<br>revised fee structures to account for higher risk profiles.<br>The temporal analysis of transactions reveals distinct patterns in payment processing. Transaction<br>volumes peak at midday, with 28.4% of daily transactions occurring between 12:00-13:00


key: chroma:document
string_value:
temporal analysis of transactions reveals distinct patterns in payment processing. Transaction<br>volumes peak at midday, with 28.4% of daily transactions occurring between 12:00-13:00 hours,<br>suggesting that payment processing systems may benefit from capacity planning around these<br>peak periods. The average transaction amount stands at €91.85, with a right-skewed distribution<br>(median €55.74), indicating that while most transactions are relatively small, a significant<br>minority involve substantial amounts exceeding €500.<br>## Fee Structure and Fee Optimization Opportunities<br>The fee structure analysis reveals complex, multi-dimensional pricing models that incorporate<br>merchant category, transaction type, and risk factors. The average fee component consists of a<br>fixed amount of €0.069 (SD €0.043) plus a variable rate of 54 basis points (SD 25.2 basis points).<br>However, these averages mask significant variation across merchant categories, with certain high-<br>risk


key: chroma:document
string_value:
of €0.069 (SD €0.043) plus a variable rate of 54 basis points (SD 25.2 basis points).<br>However, these averages mask significant variation across merchant categories, with certain high-<br>risk categories facing fees up to 89 basis points. The data suggests that approximately 32% of fee<br>rules apply universally (applicable to all merchant categories), while the remaining 68% show<br>specific category targeting.<br>The relationship between fraud rates and fee structures reveals opportunities for optimized<br>pricing. Merchants in high-fraud categories (fraud rates >15%) currently face an average fee<br>premium of 18.7 basis points compared to low-risk categories. This differential appears justified<br>by risk, but the analysis suggests potential for more granular risk-based pricing that could better<br>align fees with actual fraud exposure.<br>## Fraud Patterns and Risk Management<br>The fraud analysis reveals significant variation across merchant categories, with fraud


key: chroma:document
string_value:
pricing that could better<br>align fees with actual fraud exposure.<br>## Fraud Patterns and Risk Management<br>The fraud analysis reveals significant variation across merchant categories, with fraud rates<br>ranging from 1.2% to 18.7%. The highest fraud rates appear concentrated in specific merchant<br>categories (5812 and 5942), suggesting that these sectors may benefit from enhanced fraud<br>prevention measures. Temporal patterns in fraud show a 12% increase in fraudulent transactions<br>during holiday periods, suggesting that seasonal fraud prevention measures may be warranted.<br>The data also reveals that credit card transactions have a 23% higher fraud rate than debit card<br>transactions, highlighting the importance of differentiated fraud controls.<br>The analysis of fraudulent dispute patterns reveals that 62% of fraudulent transactions occur<br>during peak processing hours (10:00-15:00), suggesting that real-time fraud detection systems may<br>benefit from enhanced


key: chroma:document
string_value:
dispute patterns reveals that 62% of fraudulent transactions occur<br>during peak processing hours (10:00-15:00), suggesting that real-time fraud detection systems may<br>benefit from enhanced monitoring during these periods. The data also shows that merchants with<br>integrated fraud prevention systems experience 37% lower fraud rates compared to those relying<br>solely on basic controls, highlighting the value of advanced fraud prevention investments.<br>## Merchant Performance and Operational Patterns<br>The merchant performance analysis reveals significant variation in operational patterns across<br>merchant categories. The top-performing merchants by transaction volume process an average of<br>487 transactions per day, compared to the overall average of 423 transactions. Merchant category<br>7997 demonstrates particularly strong performance, with 28% higher transaction volumes than the<br>category average. The data shows that merchants with integrated payment processing


key: chroma:document
string_value:
category<br>7997 demonstrates particularly strong performance, with 28% higher transaction volumes than the<br>category average. The data shows that merchants with integrated payment processing systems<br>experience 19% higher transaction volumes compared to those using basic payment solutions.<br>The analysis of merchant category performance reveals that categories 7993 and 7372<br>demonstrate particularly strong performance metrics, with 22% and 18% higher average<br>transaction values respectively. These categories also show lower fraud rates (4.1% and 5.3%)<br>compared to the overall average of 6.3%, suggesting that certain business models may inherently<br>carry lower fraud risk. The data also reveals that merchants with dedicated payment processing<br>teams experience 27% lower chargeback rates compared to those managing payments in-house.<br>## Technical Infrastructure and System Performance<br>The technical analysis of payment processing systems reveals significant variation


key: chroma:document
string_value:
rates compared to those managing payments in-house.<br>## Technical Infrastructure and System Performance<br>The technical analysis of payment processing systems reveals significant variation in system<br>performance across merchant categories. Transactions processed through integrated payment<br>platforms experience 18% faster settlement times compared to those using basic payment<br>solutions. The data shows that merchants with API-integrated payment systems experience 23%<br>fewer failed transactions compared to those using basic payment gateways. The analysis reveals<br>that system performance varies significantly by merchant category, with category 7997<br>demonstrating particularly strong system performance metrics.<br>The analysis of system performance during peak periods reveals that merchants with load-<br>balanced payment processing systems experience 31% fewer system failures during peak hours<br>compared to those using basic infrastructure. The data shows that merchants


key: chroma:document
string_value:
that merchants with load-<br>balanced payment processing systems experience 31% fewer system failures during peak hours<br>compared to those using basic infrastructure. The data shows that merchants with real-time<br>payment processing systems experience 19% faster settlement times compared to those using<br>batch processing models. The analysis also reveals that merchants with integrated fraud<br>prevention systems experience 27% fewer chargebacks compared to those relying solely on basic<br>fraud controls.<br>## Strategic Recommendations and Conclusions<br>Based on the comprehensive analysis, we recommend that payment processors develop category-<br>specific fee structures that account for the varying risk profiles and operational characteristics of<br>different merchant categories. The data supports the development of risk-based pricing models<br>that incorporate merchant category, transaction patterns, and historical fraud rates. The analysis<br>suggests that implementing


key: chroma:document
string_value:
The data supports the development of risk-based pricing models<br>that incorporate merchant category, transaction patterns, and historical fraud rates. The analysis<br>suggests that implementing category-specific fraud prevention measures could reduce overall fraud<br>costs by an estimated 18-22%. The findings also support investment in advanced fraud prevention<br>technologies, particularly for high-risk merchant categories.<br>The analysis suggests that payment processors should develop category-specific performance<br>benchmarks to help merchants optimize their payment processing operations. The data supports<br>the development of category-specific best practice guides that address the unique challenges of<br>different merchant categories. The findings also suggest that payment processors should develop<br>category-specific support resources to help merchants optimize their payment processing<br>operations. The comprehensive nature of this analysis provides a foundation for ongoing


key: chroma:document
string_value:
should develop<br>category-specific support resources to help merchants optimize their payment processing<br>operations. The comprehensive nature of this analysis provides a foundation for ongoing research<br>into payment processing optimization and fraud prevention strategies.|Here's the report summarizing the notable trends and findings from the data analysis:<br>### 1. Payments Data Summary (payments.csv)<br>- **Size**: 23,028.7 KB (largest dataset)<br>- **Rows**: 10,000 (sampled for analysis)<br>- **Columns**: Found in dataset (specific names not available)<br>- **Findings**:<br>- Contains both numerical and categorical columns<br>- Some categorical columns have distinct value distributions<br>- Potential date/time columns identified<br>### 2. Merchant Category Codes (merchant_category_codes.csv)<br>- **Size**: 26.0 KB<br>- **Count**: Specific count not available<br>- **Findings**:<br>- Contains merchant category codes (MCC)<br>- Some merchant codes in payments data not found in


key: chroma:document
string_value:
**Size**: 26.0 KB<br>- **Count**: Specific count not available<br>- **Findings**:<br>- Contains merchant category codes (MCC)<br>- Some merchant codes in payments data not found in category file<br>### 3. Acquirer Countries (acquirer_countries.csv)<br>- **Size**: 0.2 KB<br>- **Count**: Specific count not available<br>- **Findings**:<br>- Contains country information for payment acquirers<br>- Shows distribution of top countries<br>### 4. Merchant Data (merchant_data.json)<br>- **Size**: 6.7 KB<br>- **Count**: Specific count not available<br>- **Findings**:<br>- Contains merchant information with identifiable keys<br>- Some merchant IDs in payments data not found in merchant data<br>### 5. Fees Structure (fees.json)<br>- **Size**: 518.7 KB<br>- **Record Count**: Specific count not available<br>- **Findings**:<br>- Contains fee information with identifiable keys<br>- Some acquirer IDs in fees data not found in countries file<br>### 6. Documentation<br>- **Manual**: Present (manual.md,


key: chroma:document
string_value:
**Findings**:<br>- Contains fee information with identifiable keys<br>- Some acquirer IDs in fees data not found in countries file<br>### 6. Documentation<br>- **Manual**: Present (manual.md, 21.6 KB)<br>- **Readme**: Present (payments-readme.md, 1.7 KB)<br>### Key Observations:<br>1. **Data Relationships**:<br>- Potential linkage issues between datasets (merchant IDs, category codes, and acquirer IDs)<br>- Merchant category codes in payments data don't fully match category reference file<br>- Merchant IDs in payments data don't fully match merchant data records<br>- Acquirer IDs in fees data don't fully match countries file<br>2. **Data Structure**:<br>- Payments data appears to be the core transactional dataset<br>- Supporting files provide reference information (categories, countries, merchants, fees)<br>- JSON files contain structured data while CSVs contain tabular data<br>3. **Data Quality Notes**:<br>- Inconsistencies found across different datasets<br>- Potential data


key: chroma:document
string_value:
merchants, fees)<br>- JSON files contain structured data while CSVs contain tabular data<br>3. **Data Quality Notes**:<br>- Inconsistencies found across different datasets<br>- Potential data integration challenges due to ID mismatches<br>- Documentation available to assist with interpretation<br>The analysis reveals the payments dataset as the central component, with supporting files<br>providing contextual information. Notable data integrity issues were identified regarding ID<br>matching across datasets, which should be addressed before further analysis.|Below is the summary report based solely on the available information (i.e. file names, sizes, and<br>expected content types):<br>────────────────────────────<br>Report on Data Files in Payment Dataset<br>────────────────────────────<br>1. Overview of the Dataset Files:<br>• acquirer_countries.csv (≈0.2 KB): A small CSV that likely contains metadata or mappings for<br>acquiring country codes or names.<br>• fees.json (≈518.7 KB): A


key: chroma:document
string_value:
Overview of the Dataset Files:<br>• acquirer_countries.csv (≈0.2 KB): A small CSV that likely contains metadata or mappings for<br>acquiring country codes or names.<br>• fees.json (≈518.7 KB): A moderately sized JSON file that probably provides detailed fee<br>structures, fee types, or fee calculations applicable to the payment transactions.<br>• manual.md (≈21.6 KB): A markdown document serving as a manual. This file is expected to<br>include context, instructions, or detailed explanations regarding how to work with or interpret the<br>dataset.<br>• merchant_category_codes.csv (≈26.0 KB): A CSV containing codes that classify merchants. This<br>file may help link payment transactions (or fee calculations) to business categories.<br>• merchant_data.json (≈6.7 KB): A relatively small JSON file likely containing supplementary<br>information about merchants (such as names, IDs, or basic attributes).<br>• payments-readme.md (≈1.7 KB): A short README file that probably documents the


key: chroma:document
string_value:
JSON file likely containing supplementary<br>information about merchants (such as names, IDs, or basic attributes).<br>• payments-readme.md (≈1.7 KB): A short README file that probably documents the structure,<br>source, and nuances of the payments dataset.<br>• payments.csv (≈23028.7 KB): By far the largest file in the set (over 23 MB). This appears to be<br>the core transactional dataset, probably containing detailed records of payments, such as<br>timestamps, merchant identifiers, amounts, transaction types, etc.<br>2. Notable Trends & Findings Based on File Characteristics:<br>• Main Data Concentration: The payments.csv file is significantly larger than all other files. This<br>indicates that while several supporting documents and lookup files exist, the primary volume of<br>data and most likely the majority of trends or analytics will come from analyzing this payment<br>transactions file.<br>• Structured Supporting Information: The presence of several lookup files


key: chroma:document
string_value:
of<br>data and most likely the majority of trends or analytics will come from analyzing this payment<br>transactions file.<br>• Structured Supporting Information: The presence of several lookup files (acquirer_countries.csv,<br>merchant_category_codes.csv, and merchant_data.json) suggests that the transactions in<br>payments.csv are cross-referenced against standardized metadata. This design is typical of well-<br>organized transactional databases, where linking transactions to merchant or fee information is<br>necessary.<br>• Rich Fee Data: The fees.json file, at over half a megabyte, indicates that detailed fee structures<br>(perhaps by transaction type, country, or merchant category) are maintained. Comparisons<br>between transaction records and fee specifications may reveal trends such as fee variations across<br>countries or merchant categories.<br>• Complementary Documentation: The two markdown files (manual.md and payments-<br>readme.md) indicate that the dataset is documented


key: chroma:document
string_value:
as fee variations across<br>countries or merchant categories.<br>• Complementary Documentation: The two markdown files (manual.md and payments-<br>readme.md) indicate that the dataset is documented thoroughly. Such documentation is valuable<br>in understanding the relationships between files, expected formats, and proper analytical<br>approaches.<br>3. Potential Analytical Approaches (if full file contents were to be explored):<br>• Analyzing payments.csv to extract transaction patterns (volumes over time, frequency per<br>country/merchant, seasonal spikes, etc.).<br>• Linking fees from fees.json with transactions to determine profitability, fee patterns, or cost<br>structures.<br>• Utilizing merchant_category_codes.csv and merchant_data.json to build categorical analyses<br>(e.g., trends in industries, high-activity merchant groups).<br>• Validating or cross-referencing acquirer country data with transaction records to study<br>geographic distribution or country-specific


key: chroma:document
string_value:
trends in industries, high-activity merchant groups).<br>• Validating or cross-referencing acquirer country data with transaction records to study<br>geographic distribution or country-specific behaviors.<br>• Consulting the manual.md and payments-readme.md documents to understand potential data<br>quality issues, processing instructions, or additional attributes not immediately visible from the file<br>names.<br>────────────────────────────<br>Conclusion:<br>Based on the provided file characteristics, the dataset is centered around a large volume of<br>payment transactions that are supported by detailed fee structures, merchant classifications, and<br>country mappings. The thorough documentation implies that the dataset is intended for in-depth<br>analytical studies, which might include transaction volume trends, fee impact analyses by country<br>or merchant segment, and overall payment behavior. Future analysis, when leveraging the actual<br>contents of these files, can validate


key: chroma:document
string_value:
transaction volume trends, fee impact analyses by country<br>or merchant segment, and overall payment behavior. Future analysis, when leveraging the actual<br>contents of these files, can validate these insights and uncover additional detailed findings.<br>────────────────────────────<br>This summary report relies only on file names, sizes, and inferred content types, as no direct<br>content analysis was performed.|


key: chroma:document
string_value:
_Figure 15._ A case of autonomous data science for fully open-ended data research. 

--- end of page.page_number=23 ---


key: chroma:document
string_value:
# **General Agentic Memory Via Deep Research** 

**B.Y. Yan**[1] **, Chaofan Li**[1] **, Hongjin Qian**[1] _[,]_[3] **, Shuqi Lu**[1] **, Zheng Liu**[1] _[,]_[4] _[∗]_ 

1. Beijing Academy of Artificial Intelligence 2. Renmin University of China 

3. Peking University 4. Hong Kong Polytechnic University 

```
{chienqhj,zhengliu1026}@gmail.com
```

## **Abstract**


key: chroma:document
string_value:
Memory is critical for AI agents, yet the widely-adopted static memory, aiming to create readily available memory in advance, is inevitably subject to severe information loss. To address this limitation, we propose a novel framework called **general agentic memory (GAM)** . GAM follows the principle of “ **just-in time (JIT) compilation** ” where it focuses on creating optimized contexts for its client at runtime while keeping only simple but useful memory during the offline stage. To this end, GAM employs a duo-design with the following components. 1) **Memorizer** , which highlights key historical information using a lightweight memory, while maintaining complete historical information within a universal page-store. 2) **Researcher** , which retrieves and integrates useful information from the pagestore for its online request guided by the pre-constructed memory. This design allows GAM to effectively leverage the agentic capabilities and test-time scalability of frontier large


key: chroma:document
string_value:
from the pagestore for its online request guided by the pre-constructed memory. This design allows GAM to effectively leverage the agentic capabilities and test-time scalability of frontier large language models (LLMs), while also facilitating end-to-end performance optimization through reinforcement learning. In our experimental study, we demonstrate that GAM achieves substantial improvement on various memory-grounded task completion scenarios against existing memory systems.


key: chroma:document
string_value:
## **1 Introduction** 

" _Intelligence is not the ability to store information, but to know where to find it_ ." 

## —Albert Einstein


key: chroma:document
string_value:
AI agents become increasingly popular thanks to the rapid advancement of large language models (LLMs) [1]. Today, prototypes of AI agents are being deployed across many crucial domains, such as information seeking, software engineering, and scientific research, showcasing huge potential in improving the productivity of human society [2, 3, 4]. This widespread application, however, creates an urgent need to manage complex and rapidly expanding contexts, as AI agents must continuously integrate vast amounts of information generated by both their internal reasoning and external feedback [5]. To address this challenge, there has been growing interest in developing specialized memory systems that provide agents with essential contextual information to support downstream tasks [6]. Most existing memory systems follow the principle of **Ahead-of-Time (AOT) Compilation** . Under this paradigm, substantial computation is performed during the offline stage to compress raw contexts as


key: chroma:document
string_value:
existing memory systems follow the principle of **Ahead-of-Time (AOT) Compilation** . Under this paradigm, substantial computation is performed during the offline stage to compress raw contexts as lightweight memory, while incoming requests are served primarily based on this pre-constructed memory [7, 8, 9, 10]. Although widely adopted, this AOT-style approach suffers from critical limitations.


key: chroma:document
string_value:
_**⋆ Memorization** is a form of data compression; thus, it is inevitably subject to_ _**information loss** ._ The precomputed memory, being a compressed representation of raw data, inevitably suffers from _information loss_ , making it difficult to satisfy the fine-grained information needs requested by client 

> _∗_ Project lead 

--- end of page.page_number=1 ---

**==> picture [393 x 143] intentionally omitted <==**


key: chroma:document
string_value:
> _∗_ Project lead 

--- end of page.page_number=1 ---

**==> picture [393 x 143] intentionally omitted <==**

**----- Start of picture text -----**<br>
Memorization  Deep-Research<br>q Memorizing : abstracting each page with its memo,  q Planning:  information needs (what to search),<br>e.g., { Session ID: --, Session memo: -- } searching plan (how to search)<br>q Paging : desecrating each session with its contextual  q Searching:  taking a search action, e.g., tool-using<br>information, e.g., (context info, session content) or direct browsing<br>q Reflection:  whether search results are accurate<br>page page-store and complete. If not, what’s to search next<br>Memory  is used as the working context  ⇲ Integration<br>for both Memorizer and Researcher<br>Memorizer Researcher output<br>sess 1 … sess  N memo memory Request<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
Figure 1: Overview of GAM. The memorizer generates a light memory by for agent history and keeps the complete history in the page-store during the offline stage. The researcher performs deep-research to retrieve and integrate useful information for its request in the online service. 

agents. In addition, such memory systems generally assume a _static structure_ , preventing them from flexibly adapting to ad-hoc or unforeseen requests that demand nuanced interpretation and integration of information. Finally, existing approaches often rely heavily on _domain expertise and handcrafted heuristics_ to determine how memory is constructed and organized, which further constrains generalization across domains and tasks of the AOT-style memory systems.


key: chroma:document
string_value:
_**⋆ Search** is made as the core of memory, while memorization is conducted to enable effective search._ We argue that lossless memory can only be realized via searching over a database of the complete history, where the pre-computed memory is introduced to support such a search process. With this insight, we propose **General Agentic Memory (GAM)** , a novel memory framework for general AI agents following the principle of **Just-in-Time (JIT) Compilation** . During the offline stage, it creates a light memory for the crucial historical information while maintaining the complete historical information in the database. At runtime, it performs intensive computation, namely deep research, to generate a customized, high-utility context for its request based on the pre-constructed memory. 

_**⋆ Dual-architecture**_ . Based on the above JIT principle, GAM is realized based on a dual-agent framework with two fundamental roles: the Memorizer and the Researcher (Figure 1):


key: chroma:document
string_value:
_**⋆ Dual-architecture**_ . Based on the above JIT principle, GAM is realized based on a dual-agent framework with two fundamental roles: the Memorizer and the Researcher (Figure 1): 

- The **Memorizer** receives the client’s streaming history as a sequence of sessions, where it takes two actions: 1) it dynamically compresses the key historical information with a lightweight memory, and 2) it merges each session and its corresponding memory into a page and save all pages into a page-store, ensuring that the historical information is coherently and inclusively preserved. 

- The **Researcher** receives an online request from its client and performs deep research based on the pre-constructed memory to address the client’s needs. It iteratively analyzes information need and plans search actions, retrieves relevant information from the page-store, and reflects on the results until the gathered information fully satisfies the client’s request.


key: chroma:document
string_value:
The above framework endows GAM with several key advantages. 1) _High-fidelity and taskadaptability_ , enabling the generation of concise yet highly informative memory tailored to downstream tasks. 2) _Domain generalizability_ , allowing GAM to operate effectively across general scenarios without relying on domain-specific expertise or handcrafted heuristics. 3) _Optimizability_ , harnessing advanced LLMs’ agentic capability and test-time scalability for performance optimization, while also facilitating continual improvement through reinforcement learning.


key: chroma:document
string_value:
We evaluate GAM’s performance through rigorous experimental studies. We jointly leverage the traditional memory benchmark LoCoMo [11], together with popular long-context benchmarks such as HotpotQA [7], RULER [12], and NarrativeQA [13]. Across all these experiments, GAM consistently and significantly outperforms existing methods, demonstrating its strong ability to preserve finegrained historical information and to optimize downstream task-completion performance for its clients. Our project is made publicly available to facilitate future research in this field[2] . 

> 2https://github.com/VectorSpaceLab/general-agentic-memory 

--- end of page.page_number=2 ---

## **2 Methodology** 

## **2.1 Definition**


key: chroma:document
string_value:
## **2 Methodology** 

## **2.1 Definition** 

LLM agents often require long trajectories, comprising multi-step reasoning and tool using, to accomplish complex tasks, e.g., software engineering and deep research. In our work, we define each historical trajectory (history for short) as a sequence of temporally ordered units called sessions: hist : _s_ 1 _, ..., sT_ . The rapidly growing history leads to several crucial challenges, including prohibitive computational costs, context window overflow, and performance degradation. To address these issues, a memory system is introduced to manage the information overload. Its primary objective is to extract useful yet concise information from the history, which is essential for the completion of the agent’s task. That is to say, the memory system is to optimize the _cost-effectiveness_ of the agent’s task completion grounded on its produced context. This objective can be formulated as the following min–max optimization problem.


key: chroma:document
string_value:
**Definition 2.1** ( **Memory** ) **.** A memory system produces the optimized context for an agent based on its task and history: _c[∗] ←_ Memory(task _,_ history), which is of the minimum size while optimizing the task completion performance: _c[∗]_ = argmin _C[∗] |_ c _|_ , where _C[∗]_ = argmax _C_ Agent(task _,_ context). 

## **2.2 General Agentic Memory** 

The overall architecture of GAM, depicted in Figure 1, consists of two main modules: the memorizer and the researcher. Both modules are LLM-based agents, each with customized prompts[3] , working together to generate optimized memory that addresses requests from the client agent. 

## **2.2.1 Memorizer**


key: chroma:document
string_value:
## **2.2.1 Memorizer** 

The memorizer is responsible for processing the agent’s trajectory during the offline stage, ensuring that it can be efficiently stored and effectively utilized. Each memorization step is triggered by the arrival of a new session ( _si_ ), where two operations are performed. 1. _Memorizing_ , which produces memo ( _µi_ ) as a concise and well-structured snapshot of the new session. The memo is generated based on both the new session and the existing memory ( _mi_ ), highlighting its crucial information for the entire trajectory. The memory is therefore incrementally updated with the addition of the memo: 

Memorizer _._ memorize( _si, mi_ ) _→ µi_ ; _mi_ + _{µi} → mi_ +1 _._ (1)


key: chroma:document
string_value:
Memorizer _._ memorize( _si, mi_ ) _→ µi_ ; _mi_ + _{µi} → mi_ +1 _._ (1) 

2. _Paging_ , which creates pages to maintain the complete information of the agent’s trajectory. It begins by generating a header for the new session, which contains crucial contextual information from its preceding trajectory. The header is then used to decorate the session, forming a new page that is subsequently added to the page-store ( _p_ ): 

Memorizer _._ page( _si, mi_ ) _→ hi_ ; _{_ header : _hi,_ content : _si} → pi_ ; _p._ append( _pi_ ) _._ (2) 

This process shares the same principle of BGE landmark retrieval [14] and Anthropic contextual retrieval [15], which preserve the consistency of page semantics, ensuring that they can be accurately retrieved in subsequent stages. 

## **2.2.2 Researcher**


key: chroma:document
string_value:
## **2.2.2 Researcher** 

The researcher is to address the client’s request by retrieving and integrating useful information from the page-store. The process is iteratively conducted with three operations. 1) _Planning_ , which performs a chain-of-thought reasoning based on the existing memory to analyze the underlying information needed by request ( _r_ ). Based on this initial reasoning result, it further generates concrete search plans according to the provided search toolkit ( _T_ ): 

Researcher _._ plan( _r, mi, T_ ) _→{_ tool : _t_ ; parameter : _ρt}t∈T ._ (3) 

In our implementation, we offer three available tools for the researcher: an embedding model for vector search, a BM25 retriever for keyword-based search, and an ID-based retriever for direct page exploration. 2) _Searching_ . Upon obtaining the search plan, the researcher executes each search action in parallel, retrieving relevant pages ( _pt_ ) from the page-store. The researcher then integrates the


key: chroma:document
string_value:
> 3We include the detailed prompts of all functions in the appendix of the paper. 

--- end of page.page_number=3 ---

information from the union of the retrieved pages together with the last integration result ( _I_ ) for the request ( _r_ ), leading to an updated temporal integration result: 

For each _t_ : _t_ ( _ρt_ ) _→ pt_ ; Researcher _._ integrate(� _t∈T[p][t][,][ I][, r]_[)] _[ →I][.]_ (4) 

3) _Reflection_ . The researcher performs a reflection on whether the needed information in the request ( _r_ ) has been fully collected by the integration result _I_ using a binary indicator ( _y_ ). If no, it further analyzes for the missing information, leading to a new request _r[′]_ to drive another round of deep research. If yes, the research process will be concluded by returning the integration result: 

Researcher _._ reflect( _I, r_ ) _→ y, r[′]_ ; if _y_ = No, Researcher( _r[′] , I_ ); if _y_ = Yes, return _I._ (5)


key: chroma:document
string_value:
Researcher _._ reflect( _I, r_ ) _→ y, r[′]_ ; if _y_ = No, Researcher( _r[′] , I_ ); if _y_ = Yes, return _I._ (5) 

Finally, the integrated result, along with the original information extracted from the associated pages, is returned to the client as the optimized context for its downstream task completion. 

## **2.2.3 Optimization** 

A unified end-to-end performance optimization framework is introduced for GAM. Suppose a training dataset _D_ = _{_ (task _,_ hist) _}_ is given, the system creates the memory and page-store as: M _,_ P _←_ Memorizer(hist), and then generates a candidate context for the task via: _c ←_ Researcher(task _,_ M _,_ P). Using this candidate context, the client samples an answer (ans), whose _·_ quality is measured by the reward function Γ( ). Thus, the expected reward is derived as: 

_R_ = Etask _,_ hist _∼D_ EM _,_ P _∼_ Memorizer(hist) E _c∼_ Researcher(task _,_ M _,_ P) Eans _∼_ Client( _c,_ task) Γ(ans) _._ (6)


key: chroma:document
string_value:
_R_ = Etask _,_ hist _∼D_ EM _,_ P _∼_ Memorizer(hist) E _c∼_ Researcher(task _,_ M _,_ P) Eans _∼_ Client( _c,_ task) Γ(ans) _._ (6) 

When focusing on optimizing GAM’s performance, the memorizer and the researcher are learned via reinforcement, while the client is excluded from the learning process. Without loss of generality, the policy gradients for the memorizer and researcher are given by: 

**==> picture [246 x 14] intentionally omitted <==**

> _,_ (7) _∇θr_ = Etask _,_ hist _∼D_ (Γ(ans) _−_ Γ[¯] _r_ ) _∇θr_ log _πr_ (c _|_ task _,_ M _,_ P) _._ 

Here, _θm_ and _θr_ denote the model parameters of memorizer and researcher, respectively; Γ[¯] _m_ and Γ[¯] _r_ are the baseline answer rewards of the two modules; while _θm_ ( _·_ ) and _θr_ ( _·_ ) stand for the memorizer and researcher’s generation likelihood. 

## **3 Experiment**


key: chroma:document
string_value:
## **3 Experiment** 

In this section, we conduct comprehensive experimental studies to evaluate the effectiveness of GAM. We focus on the investigation of the following three research questions. **RQ 1** : How does GAM perform compared with existing memory systems? **RQ 2** : How does GAM’s performance vary across different scenarios? **RQ 3** : How do key technical factors within GAM influence its performance? 

## **3.1 Experiment Setting**


key: chroma:document
string_value:
**Datasets.** To rigorously evaluate the effectiveness of GAM, specifically 1) the memory’s ability to preserve historical information and 2) its ability to support downstream task completion, we employ the following benchmarks in our experimental studies. **1) LoCoMo** [11]. A widely used memory benchmark for conversational settings, designed to evaluate an agent’s ability to maintain and recall information across extended multi-session dialogues. We adopt its single-hop, multi-hop, temporal-reasoning, and open-domain tasks in our experiments. **2) HotpotQA** [16]. A popular multi-hop question answering benchmark based on the Wikipedia corpus. We use the curated memory-evaluation dataset in MemAgent [7] that concatenates gold supporting documents with distracting passages. By varying the number of distractions, the dataset provides three versions with context lengths of 56K, 224K, and 448K tokens. **3) RULER** [12]. A popular long-context understanding benchmark with four types of


key: chroma:document
string_value:
number of distractions, the dataset provides three versions with context lengths of 56K, 224K, and 448K tokens. **3) RULER** [12]. A popular long-context understanding benchmark with four types of evaluation tasks, including retrieval (Retri.), multi-hop tracing (MT), aggregation (AGG.), and question answering (QA). We use the 128K-token setting in our experiments. **4) NarrativeQA** [13]. A long-context question answering benchmark that provides an entire book or movie script as the input context for each sample. We randomly sample a subset of 300 questions for evaluation, whose average token length is 87K.


key: chroma:document
string_value:
**Baselines.** We consider the following baselines in our experiments. **1) Memory-free methods** , including the _brute-force long-LLM_ (long-LLM for brevity) and retrieval-augmented generation (RAG). The long-LLM baseline attempts to process the entire input within the model’s context window. When the number of input tokens exceeds the maximum allowable context length _Lmax_ , the input is evenly partitioned into _N_ chunks of length _Lmax_ : _{S_ 1 _, ..., SN }_ , where the final score is reported as the maximum over all chunks: max _{_ LLM( _S_ 1) _..._ LLM( _SN_ ) _}_ . For the RAG baseline, the input is uniformly partitioned into segments of 2,048 tokens, and the top-5 retrieved segments are used to perform the downstream task. **2) Memory-based methods** , including A-Mem [8], Mem0 [9], MemoryOS [10] and LightMem [17]. These approaches construct specialized memory structures to store historical information, which can be utilized to address memory-related tasks at runtime.


key: chroma:document
string_value:
**Implementation Details.** In our experiments, we adopt GPT-4o-mini and Qwen2.5-14B-Instruct [18] as the backbone models for both GAM and all baselines. Both LLMs offer a long-context window of 128K tokens. We use BGE-M3 [19] as the default dense retriever. For GAM’s detailed configuration, we set the maximum reflection depth to 3 and the maximum number of retrieved pages to 5. The input context is segmented into 2,048-token pages for stream processing in the memorization module. 

## **3.2 Main Results: Overall Effectiveness**


key: chroma:document
string_value:
Table 1 presents the main results of GAM and baselines on the experimental benchmarks, from which the following observations can be made. First, GAM consistently outperforms all baselines, including both memory-free and memory-based approaches, across every benchmark. Moreover, its advantage becomes particularly pronounced on benchmarks like HotpotQA and RULER, where tasks require multi-step retrieval and reasoning over information dispersed within the input context. For instance, GAM achieves over 90% accuracy on the multi-hop tracing (MT) tasks in the RULER benchmark, which demand tracking variable values across multiple steps of assignment; in contrast, most baselines fail to achieve satisfactory performance under such complexity. Finally, GAM maintains stable and competitive performance under varying input-context lengths, as reflected in the results on HotpotQA. In summary, these experimental results preliminarily verify GAM’s overall effectiveness and its robustness to task


key: chroma:document
string_value:
under varying input-context lengths, as reflected in the results on HotpotQA. In summary, these experimental results preliminarily verify GAM’s overall effectiveness and its robustness to task complexity and growing context lengths.


key: chroma:document
string_value:
We obtain the following interesting things besides the main observations. First, the performance of long-LLMs is under-expectation compared with the other methods, despite that it has adopted LLMs with a 128K context window, long enough to fully cover the input context in LoCoMo, HotpotQA56K, and NarrativeQA. This suggests that simply extending the context window is insufficient to effectively address long-context challenges. This also aligns with the recently discussed phenomenon of context rot[4] , which indicates that the substantial distracting or irrelevant information within long contexts can severely degrade LLMs’ performance. Second, direct applications of retrieval, i.e., RAG, exhibit highly variable performance across different scenarios. RAG improves performance over long-LLMs and the memory-based methods when the relevant information is explicitly presented, such as LoCoMo single-hop and RULER retrieval. However, it performs badly in HotpotQA, RULER multi-hop tracing, and


key: chroma:document
string_value:
and the memory-based methods when the relevant information is explicitly presented, such as LoCoMo single-hop and RULER retrieval. However, it performs badly in HotpotQA, RULER multi-hop tracing, and RULER aggregation tasks, where relevant information is unobvious. In comparison, the memory-based methods show lower variance but remain constrained due to the loss of crucial details of the original context. In contrast, GAM leverages memory to support effective retrieval of task-relevant information, enabling it to achieve substantially improved performance.


key: chroma:document
string_value:
## **3.3 Model’s Impact** 

Table 2 presents the performance of GAM on HotpotQA and NarrativeQA implemented with different LLMs. We apply Qwen-2.5 variants of different sizes (from 0.5B to 32B) and GPT-4o-mini as the backbones of the memorization and research module. As demonstrated by the experiment result, larger and stronger LLM-backbones for both memorizer and researcher result in consistent performance improvement, indicating that GAM can effectively leverage the increased LLM capacity to improve its memory quality. However, we also observe that the research module is considerably more sensitive to the LLM’s scale than the memorization module. Notably, GAM maintains strong performance even when the memorizer is downsized and remains competitive with the smallest Qwen-2.5-0.5B model. In contrast, GAM’s overall performance deteriorates significantly when the research module’s backbone is reduced to 7B or smaller. This discrepancy reflects the distinct complexity of the two


key: chroma:document
string_value:
> 4https://research.trychroma.com/context-rot 

--- end of page.page_number=5 ---

Table 1: Results from GAM and baselines (memory-free and memory-based) on LoCoMo, HotpotQA, RULER, and NarrativeQA. Two LLMs, GPT-4o-mini and Qwen-2.5-14B, are used in experiment. 

(a) Results on LoCoMo.


key: chroma:document
string_value:
|**Model**||**Method**|||||**LoCoMo**|**LoCoMo**|**LoCoMo**|**LoCoMo**|**LoCoMo**|**LoCoMo**|**LoCoMo**|**LoCoMo**|**LoCoMo**||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
||||**Single Hop**||||**Multi Hop**|||||**Temporal**||||**Open Domain**|||
||||**F1**<br>**BLEU-1**||||**F1**<br>**BLEU-1**|||||**F1**<br>**BLEU-1**||||**F1**<br>**BLEU-1**|||
|**GPT-4o-mini**||LONG-LLM<br>RAG|46.68<br>52.45||37.54<br>47.94||29.23<br>27.50|||22.76<br>20.13||25.97<br>46.07||19.42<br>40.35||16.87<br>23.23||13.70<br>17.94|
|||A-MEM<br>MEM0<br>MEMORYOS<br>LIGHTMEM|44.65<br>47.65<br>48.62<br>41.79||37.06<br>38.72<br>42.99<br>37.83||27.02<br>38.72<br>35.27<br>29.78|||20.09<br>27.13<br>25.22<br>24.80||45.85<br>48.93<br>41.15<br>43.71||36.67<br>40.51<br>30.76<br>39.72||12.14<br>28.64<br>20.02<br>16.89||12.00<br>21.58<br>16.52<br>13.92|
|||GAM|**57.75**||**52.10**||**42.29**|||**34.44**||**59.45**||**53.11**||**33.30**||**26.97**|


key: chroma:document
string_value:
|||GAM|**57.75**||**52.10**||**42.29**|||**34.44**||**59.45**||**53.11**||**33.30**||**26.97**|
|**Qwen2.5 14b**||LONG-LLM<br>RAG|46.05<br>47.87||39.56<br>42.79||32.08<br>26.38|||24.46<br>19.54||30.51<br>30.78||24.45<br>25.97||14.89<br>14.16||11.41<br>10.52|
|||A-MEM<br>MEM0<br>MEMORYOS<br>LIGHTMEM|33.75<br>42.58<br>46.33<br>34.92||30.04<br>35.15<br>41.62<br>31.22||22.09<br>31.73<br>38.19<br>25.45|||15.28<br>24.82<br>29.26<br>19.61||27.19<br>28.96<br>32.24<br>32.03||22.05<br>26.24<br>27.86<br>27.70||13.49<br>15.03<br>20.27<br>15.81||10.74<br>11.28<br>15.94<br>11.81|
|||GAM|**58.93**||**53.74**||**42.96**|||**34.48**||**51.52**||**44.43**||**30.63**||**26.04**|
|**Model**<br>**GPT-4o-mini**<br>**Qwen2.5 14b**|||(b) Results on HotpotQA, RULER, and NarrativeQA.||||||||||||||||
|||**Method**||**HotpotQA**|||||||**RULER(128k)**|||||**NarrativeQA**|||


key: chroma:document
string_value:
|**Model**<br>**GPT-4o-mini**<br>**Qwen2.5 14b**|||(b) Results on HotpotQA, RULER, and NarrativeQA.||||||||||||||||
|||**Method**||**HotpotQA**|||||||**RULER(128k)**|||||**NarrativeQA**|||
|||||**56K**<br>**F1**|**224K**<br>**F1**|**448K**<br>**F1**|||**Retri.**<br>**Acc.**||**MT**<br>**Acc.**||**AGG.**<br>**Acc.**||**QA**<br>**Acc.**||F1||
|||LONG-LLM<br>RAG||56.56<br>52.71|54.29<br>51.84|53.92<br>54.01|||80.30<br>94.25||60.60<br>0.00||36.70<br>35.50||61.60<br>55.90||31.26<br>25.00||
|||A-MEM<br>MEM0<br>MEMORYOS<br>LIGHTMEM||33.90<br>32.58<br>26.47<br>40.93|30.22<br>31.74<br>23.10<br>35.28|31.37<br>27.41<br>24.16<br>30.02|||44.23<br>46.83<br>63.10<br>27.63||0.00<br>53.80<br>2.40<br>36.20||29.20<br>34.10<br>35.60<br>34.00||46.50<br>51.70<br>36.90<br>52.60||27.07<br>29.16<br>26.70<br>17.51||
|||GAM||**63.22**|**64.56**|**59.81**|||**97.70**||**93.20**||**42.50**||**72.50**||**36.86**||


key: chroma:document
string_value:
|||GAM||**63.22**|**64.56**|**59.81**|||**97.70**||**93.20**||**42.50**||**72.50**||**36.86**||
|||LONG-LLM<br>RAG||49.75<br>51.81|46.82<br>46.72|43.17<br>48.36|||70.85<br>92.78||80.00<br>0.00||15.40<br>24.70||45.60<br>47.80||29.69<br>18.29||
|||A-MEM<br>MEM0<br>MEMORYOS<br>LIGHTMEM||27.04<br>30.12<br>24.58<br>37.30|25.65<br>32.44<br>30.25<br>27.72|22.92<br>26.55<br>23.13<br>28.25|||39.73<br>43.03<br>54.58<br>27.53||0.00<br>41.20<br>3.00<br>17.40||25.80<br>31.50<br>5.20<br>25.60||40.20<br>46.10<br>34.60<br>53.00||25.18<br>27.80<br>23.45<br>16.57||
|||GAM||**64.07**|**55.99**|**57.87**|||**93.43**||**90.20**||**36.10**||**74.50**||**34.77**||


key: chroma:document
string_value:
modules: the memorizer primarily extracts salient information from the input context, which is a relatively straightforward task, whereas the researcher must conduct iterative planning, searching, and reflection, which is much more complex and thus demands greater model capacity. 

## **3.4 Increasing Test-Time Computation** 

As shown in Figure 2, we investigate the impact of increasing test-time computation from two perspectives: 1) the depth of reflection and 2) the amount of retrieved pages. First, we vary the maximum reflection depth from 1 to 5 (3 by default), allowing GAM to perform additional research steps when necessary. Note that GAM autonomously determines the actual number of reflections 

--- end of page.page_number=6 ---

Table 2: Model’s impact on memorizer (left) and researcher (right), reflected by GAM’s performance. 

**==> picture [396 x 100] intentionally omitted <==**


key: chroma:document
string_value:
--- end of page.page_number=6 ---

Table 2: Model’s impact on memorizer (left) and researcher (right), reflected by GAM’s performance. 

**==> picture [396 x 100] intentionally omitted <==**

**----- Start of picture text -----**<br>
(a) Memorizer (b) Researcher<br>HotpotQA NarrativeQA Avg HotpotQA NarrativeQA Avg<br>Model 56K 224K 448K Model 56K 224K 448K<br>F1 F1 F1 F1<br>F1 F1 F1 F1 F1 F1<br>QWEN2.5 0.5B 56.46 55.96 53.33 29.55 48.83 QWEN2.5 0.5B 10.03 11.14 11.64 3.50 9.08<br>QWEN2.5 3B 58.05 56.52 55.50 32.10 50.54 QWEN2.5 3B 39.76 37.16 33.04 23.96 33.48<br>QWEN2.5 7B 59.06 58.34 56.17 32.53 51.53 QWEN2.5 7B 51.95 47.95 48.55 26.93 43.85<br>QWEN2.5 14B 64.07 55.99 57.87 34.77 53.18 QWEN2.5 14B 64.07 55.99 57.87 34.77 53.18<br>QWEN2.5 32B 63.05 59.75 56.26 34.94 53.50 QWEN2.5 32B 61.93 59.19 61.53 35.33 54.50<br>GPT-4O MINI 64.77 59.29 57.25 34.87 54.05 GPT-4O-MINI 62.06 62.97 61.54 35.24 55.45<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
**==> picture [397 x 100] intentionally omitted <==**

**==> picture [397 x 137] intentionally omitted <==**

**----- Start of picture text -----**<br>
(a) Impact of maximum reflection depth.<br>(b) Impact of the amount of retrieved pages.<br>**----- End of picture text -----**<br>


Figure 2: Impact of increasing test-time computation in reflection (top) and retrieval (bottom).


key: chroma:document
string_value:
Figure 2: Impact of increasing test-time computation in reflection (top) and retrieval (bottom). 

and does not always reach the maximum step. This increased flexibility enables GAM to collect more information from the page-store, thus yielding consistent performance improvements across all datasets. However, the marginal gains gradually diminish, as many tasks do not require deep multi-step reasoning. Second, we increase the number of retrieved pages from 3 to 20 (5 by default), enabling GAM to browse more pages in each step of research. The increase in retrieval results also leads to consistent performance improvements. Overall, both forms of increased test-time computation result in steady performance gains, which demonstrates GAM’s ability to benefit from test-time scaling, an advantage that baseline methods lack due to their fixed workflows. 

## **3.5 Detailed Factors’ Analysis**


key: chroma:document
string_value:
## **3.5 Detailed Factors’ Analysis** 

We perform ablation studies to analyze other detailed influential factors, including searching tools, formation of GAM, and output formats. 

First, we examine the impact of each searching tool and its combinations. As shown in Table 3, combining any two of the searching tools yields better performance than using each single tool alone, and the joint use of all three tools (i.e., GAM with the default setting) achieves the best performance. This observation validates the effectiveness of the search tools. Moreover, employing multiple tools enables broader exploration of the page-store, leading to better coverage of relevant information and, consequently, improved performance. 

--- end of page.page_number=7 ---

Table 3: Ablation study of detailed factors.


key: chroma:document
string_value:
--- end of page.page_number=7 ---

Table 3: Ablation study of detailed factors. 

|**Method**|**HotpotQA**|**NarrativeQA**|**Avg**|
|---|---|---|---|
||**56K**<br>**224K**<br>**448K**<br>**F1**<br>**F1**<br>**F1**|**F1**|**F1**|
|GAM|64.07<br>55.99<br>57.87|34.77|53.18|
|**Tools**||||
|ONLY PAGE-ID<br>ONLY EMBEDDING<br>ONLY BM25<br>EMBEDDING+PAGE-ID<br>EMBEDDING+BM25<br>BM25+PAGE-ID|44.86<br>21.65<br>19.02<br>39.59<br>32.71<br>26.67<br>59.24<br>52.29<br>51.52<br>47.25<br>34.78<br>28.43<br>61.37<br>55.00<br>54.90<br>63.57<br>55.38<br>55.62|30.30<br>30.25<br>31.50<br>33.41<br>33.20<br>32.05|28.96<br>32.31<br>48.64<br>35.97<br>51.12<br>51.66|
|**Modules**||||
|RESEARCH WITHOUT MEMORY<br>MEMORY WITHOUT RESEARCH|57.40<br>49.72<br>53.98<br>42.67<br>19.75<br>17.38|31.97<br>30.18|48.27<br>27.50|



Table 4: Performance across different output formats.


key: chroma:document
string_value:
Table 4: Performance across different output formats. 

|**Model**|**Metric**|**HotpotQA**|**NarrativeQA**|**Avg**|
|---|---|---|---|---|
|||**56K**<br>**224K**<br>**448K**|||
|INTEGRATION ONLY|F1<br>Tokens|64.07<br>55.99<br>57.87<br>103.42<br>102.55<br>109.98|34.77<br>107.64|53.18<br>105.90|
|INTEGRATION WITH PAGE|F1<br>Tokens|68.66<br>59.77<br>59.42<br>1444.30<br>499.23<br>620.11|34.99<br>6955.62|55.71<br>2379.82|
|INTEGRATION WITH EXTRACTION|F1<br>Tokens|67.41<br>57.83<br>57.81<br>220.78<br>227.57<br>230.47|34.82<br>244.20|54.47<br>230.76|


key: chroma:document
string_value:
Second, we evaluate GAM’s performance when each module is used in isolation, namely 1) research without memory, and 2) memory without research. According to the experiment result in Table 3, using the research module alone leads to a substantial performance drop compared with the complete GAM system, highlighting the crucial role of memory in supporting effective exploration of relevant information. Using the memory module alone results in even worse performance, indicating that the pre-computed memory is prone to severe information loss. This observation further echoes our previous conclusion that the pre-constructed memory used in traditional ahead-of-time paradigms is far more limited than the just-in-time approach adopted by GAM.


key: chroma:document
string_value:
Third, we explore the impact of different forms of output, including 1) the researcher’s integration result (default), 2) the integration result accompanied by the relevant pages that provided its source information, and 3) the integration result paired with extracted source snippets from those relevant pages. As shown in Table 4, using only the integration result already achieves highly competitive performance. However, augmenting it with source information from the relevant pages yields further improvements, as it helps mitigate the loss of fine-grained details that may occur during integration. 

## **3.6 Efficiency**


key: chroma:document
string_value:
## **3.6 Efficiency** 

To assess the working efficiency of GAM, we measure the average time consumption, including both offline memory construction and online serving, when processing HotpotQA tasks under the 56K, 224K, and 448K settings. As shown in Table 5, GAM incurs a time cost comparable to Mem0 and MemoryOS, and is substantially faster than A-mem. All methods exhibit approximately linear growth in offline construction time as context length increases, while maintaining relatively stable online serving time. Overall, GAM delivers strong performance with competitive efficiency, offering the best cost-effectiveness among experimental approaches. 

--- end of page.page_number=8 ---

Table 5: Efficiency analysis on HotpotQA


key: chroma:document
string_value:
Table 5: Efficiency analysis on HotpotQA 

|**Dataset**|**Metric**<br>**A-mem**<br>**Mem0**<br>**MemoryOS**<br>**LightMem**<br>**GAM**|
|---|---|
|**HotpotQA 56k**|OFFLINE BUILD (s)<br>209.74<br>37.42<br>80.36<br>4.93<br>56.89<br>ONLINE SERVE (s)<br>0.52<br>0.15<br>0.44<br>0.20<br>12.43<br>TOTAL (s)<br>210.26<br>37.57<br>80.80<br>5.13<br>69.32|
||ANSWER QUALITY (F1)<br>27.04<br>30.12<br>24.58<br>37.30<br>64.07|
|**HotpotQA 224k**|OFFLINE BUILD (s)<br>904.99<br>165.30<br>325.70<br>16.61<br>252.72<br>ONLINE SERVE (s)<br>0.48<br>0.17<br>0.55<br>0.25<br>16.65<br>TOTAL (s)<br>905.46<br>165.47<br>326.25<br>16.86<br>269.37|
||ANSWER QUALITY (F1)<br>25.65<br>32.44<br>30.25<br>27.72<br>55.99|
|**HotpotQA 448k**|OFFLINE BUILD (s)<br>1796.82<br>274.87<br>702.72<br>40.56<br>557.16<br>ONLINE SERVE (s)<br>0.47<br>0.18<br>0.46<br>0.21<br>18.49<br>TOTAL (s)<br>1797.29<br>275.05<br>703.18<br>40.78<br>575.65|
||ANSWER QUALITY (F1)<br>22.92<br>26.55<br>23.13<br>28.25<br>57.87|



## **4 Conclusion**


key: chroma:document
string_value:
## **4 Conclusion** 

In this paper, we present a novel memory system called General Agentic Memory (GAM), which is developed under the just-in-time compilation principle. GAM employs a dual-framework comprising a memorizer and a researcher. During the offline stage, the memorizer extracts the key information for its incoming context with lightweight memory and preserve the complete information within a page-store. For each online request, the researcher performs deep-research over the page-store based on the pre-constructed memory, which generates concise yet informative memory to support the downstream task. We perform comprehensive empirical studies using a variety of popular memory and long-context benchmarks, whose result validates the effectiveness of GAM given its significant and consistent improvements over existing methods. 

## **References**


key: chroma:document
string_value:
## **References** 

- [1] Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, Zihao Wang, Zekai Wang, Feng Yin, Junhua Zhao, et al. Exploring large language model based intelligent agents: Definitions, methods, and prospects. _arXiv preprint arXiv:2401.03428_ , 2024. 

- [2] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. _Advances in Neural Information Processing Systems_ , 36:28091–28114, 2023. 

- [3] Wei Tao, Yucheng Zhou, Yanlin Wang, Wenqiang Zhang, Hongyu Zhang, and Yu Cheng. Magis: Llm-based multi-agent framework for github issue resolution. _Advances in Neural Information Processing Systems_ , 37:51963–51993, 2024. 

- [4] Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Michael Moor, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. _arXiv preprint arXiv:2501.04227_ , 2025.


key: chroma:document
string_value:
- [5] Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. _arXiv preprint arXiv:2306.15595_ , 2023. 

- [6] Zeyu Zhang, Quanyu Dai, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model-based agents. _ACM Transactions on Information Systems_ , 43(6):1–47, 2025. 

- [7] Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, et al. Memagent: Reshaping long-context llm with multi-conv rl-based memory agent. _arXiv preprint arXiv:2507.02259_ , 2025. 

--- end of page.page_number=9 ---

- [8] Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. _arXiv preprint arXiv:2502.12110_ , 2025.


key: chroma:document
string_value:
--- end of page.page_number=9 ---

- [8] Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. _arXiv preprint arXiv:2502.12110_ , 2025. 

- [9] Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory. _arXiv preprint arXiv:2504.19413_ , 2025. 

- [10] Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory os of ai agent. _arXiv preprint arXiv:2506.06326_ , 2025. 

- [11] Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of llm agents. _arXiv preprint arXiv:2402.17753_ , 2024. 

- [12] Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. Ruler: What’s the real context size of your long-context language models? _arXiv preprint arXiv:2404.06654_ , 2024.


key: chroma:document
string_value:
- [13] Tomáš Koˇcisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor` Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. _Transactions of the Association for Computational Linguistics_ , 6:317–328, 2018. 

- [14] Kun Luo, Zheng Liu, Shitao Xiao, Tong Zhou, Yubo Chen, Jun Zhao, and Kang Liu. Landmark embedding: a chunking-free embedding method for retrieval augmented long-context large language models. In _Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 3268–3281, 2024. 

- [15] Anthropic. Introducing contextual retrieval. _https://www.anthropic.com/engineering/contextualretrieval_ , 2024. 

- [16] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. _arXiv preprint arXiv:1809.09600_ , 2018.


key: chroma:document
string_value:
- [17] Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, et al. Lightmem: Lightweight and efficient memoryaugmented generation. _arXiv preprint arXiv:2510.18866_ , 2025. 

- [18] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. _arXiv preprint arXiv:2309.16609_ , 2023. 

- [19] Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. _arXiv preprint arXiv:2402.03216_ , 2024. 

--- end of page.page_number=10 ---

## **Appendix** 

## **Baseline Reproduction Details**


key: chroma:document
string_value:
--- end of page.page_number=10 ---

## **Appendix** 

## **Baseline Reproduction Details** 

When reproducing the baseline methods on the LocoMo dataset, we found that the category labels used for A-mem, Mem0, and MemoryOS were incorrect. Based on the official LocoMo annotations, we corrected the corresponding category–label mapping. 

## **Prompts**


key: chroma:document
string_value:
## **Prompts** 

**Prompt for Memorizing** You are the MemoryAgent. Your job is to write one concise abstract that can be stored as long-term memory. MAIN OBJECTIVE: Generate a concise, self-contained and coherent abstract of INPUT_MESSAGE that preserves ALL important information in INPUT_MESSAGE. MEMORY_CONTEXT is provided so you can understand the broader situation such as people, modules, decisions, ongoing tasks and keep wording consistent. INPUTS: **MEMORY_CONTEXT** : {memory_context} **INPUT_MESSAGE** : {input_message} YOUR TASK: 1. Read INPUT_MESSAGE and extract all specific, memory-relevant information, such as: - plans, goals, decisions, requests, preferences 

- actions taken, next steps, assignments, and responsibilities 

- problems, blockers, bugs, questions that need follow-up - specific facts such as names, dates, numbers, locations 2. Use MEMORY_CONTEXT to: - resolve or disambiguate the entities, components, tasks, or resources mentioned in INPUT_MESSAGE,


key: chroma:document
string_value:
- keep terminology (names of agents, modules, datasets, etc.) consistent with prior usage, - include minimal background context if it is required for the abstract to be understandable. You MUST NOT invent or add information that appears only in MEMORY_CONTEXT and is NOT implied or mentioned in INPUT_MESSAGE. 3. Your abstract MUST: 

- summarize all important content from INPUT_MESSAGE, 

- be understandable on its own without seeing INPUT_MESSAGE, 

- be factual and specific. 

STYLE RULES: - Output exactly ONE concise paragraph. No bullet points. - Do NOT include meta phrases like "The user said..." or "The conversation is about...". 

- Do NOT give advice, opinions, or suggestions. 

- Do NOT ask questions. - Do NOT include anything that is not grounded in INPUT_MESSAGE. OUTPUT FORMAT: Return ONLY the single paragraph. Do NOT add any headings or labels. 

--- end of page.page_number=11 ---

## **Prompt for Planning Part 1**


key: chroma:document
string_value:
--- end of page.page_number=11 ---

## **Prompt for Planning Part 1** 

You are the PlanningAgent. Your job is to generate a concrete retrieval plan for how to gather information needed to answer the QUESTION. You must use the QUESTION and the current MEMORY (which contains abstracts of all messages so far). **QUESTION** : {request} **MEMORY** : {memory}


key: chroma:document
string_value:
PLANNING PROCEDURE 1. Interpret the QUESTION using the context in MEMORY. Identify what is need to satisfy the QUESTION. 2. Break that need into concrete "info needs": specific sub-questions you must answer to fully respond to the QUESTION. 3. For each info need, decide which retrieval tools are useful. You may assign multiple tools to the same info need: - Use "keyword" for exact entities / functions / key attributes. - Use "vector" for conceptual understanding. - Use "page_index" if MEMORY already points to clearly relevant page indices. 4. Build the final plan: - "info_needs": a list of all the specific sub-questions / missing facts you still need. - "tools": which of ["keyword","vector","page_index"] you will actually use in this plan. This can include more than one tool. - "keyword_collection": a list of short keyword-style queries you will issue. - "vector_queries": a list of semantic / natural-language queries you will issue. - "page_index": a list of integer page indices you


key: chroma:document
string_value:
a list of short keyword-style queries you will issue. - "vector_queries": a list of semantic / natural-language queries you will issue. - "page_index": a list of integer page indices you plan to read fully.


key: chroma:document
string_value:
AVAILABLE RETRIEVAL TOOLS: All of the following retrieval tools are available to you. You may select one, several, or all of them in the same plan to maximize coverage. Parallel use of multiple tools is allowed and encouraged if it helps answer the QUESTION. 1. "keyword" - WHAT IT DOES: Exact keyword match retrieval. It finds pages that contain specific names, function names, key attributes, etc. - HOW TO USE: Provide short, high-signal keywords. Do NOT write long natural-language questions here. Use crisp keywords and phrases that should literally appear in relevant text. 2. "vector" - WHAT IT DOES: Semantic retrieval by meaning. It finds conceptually related pages. This is good for high-level questions, reasoning questions, or "how/why" style questions. - HOW TO USE: Write each query as a short natural-language sentence that clearly states what you want to know, using full context and entities from MEMORY and QUESTION. Example style: "How does the DenseRetriever assign GPUs during


key: chroma:document
string_value:
a short natural-language sentence that clearly states what you want to know, using full context and entities from MEMORY and QUESTION. Example style: "How does the DenseRetriever assign GPUs during index building?"


key: chroma:document
string_value:
**Prompt for Planning Part 2** 3. "page_index" - WHAT IT DOES: Directly ask to re-read full pages (by page ID) that are already known to be relevant. MEMORY may mention specific page IDs or indices that correspond to important configs, attributes, or names. Use this if you already know specific page indices that should be inspected in full. - HOW TO USE: Return a list of those integer page indices (e.g. [0, 2, 5]), max 5 pages. You MUST NOT invent or guess page indices. RULES - Avoid simple repetition. Whether it's keywords or sentences for search, make them as independent as possible rather than duplicated. - Be specific. Avoid vague items like "get more details" or "research background". - Every string in "keyword_collection" and "vector_queries" must be directly usable as a retrieval query. - You may include multiple tools. Do NOT limit yourself to a single tool if more than one is useful. - Do NOT invent tools. Only use "keyword", "vector", "page_index". - Do NOT invent page


key: chroma:document
string_value:
query. - You may include multiple tools. Do NOT limit yourself to a single tool if more than one is useful. - Do NOT invent tools. Only use "keyword", "vector", "page_index". - Do NOT invent page indices. If you are not sure about a page index, return []. - You are only planning retrieval. Do NOT answer the QUESTION here. THINKING STEP - Before producing the output, think through the procedure and choices inside <think>...</think>. - Keep the <think> concise but sufficient to validate decisions. - After </think>, output ONLY the JSON object specified below. The <think> section must NOT be included in the JSON. OUTPUT JSON SPEC Return ONE JSON object with EXACTLY these keys: - "info_needs": array of strings (required) - "tools": array of strings from ["keyword","vector","page_index"] (required) - "keyword_collection": array of strings (required) - "vector_queries": array of strings (required) - "page_index": array of integers (required), max 5. All keys MUST appear. After the <think>


key: chroma:document
string_value:
- "keyword_collection": array of strings (required) - "vector_queries": array of strings (required) - "page_index": array of integers (required), max 5. All keys MUST appear. After the <think> section, return ONLY the JSON object. Do NOT include any commentary or explanation outside the JSON.


key: chroma:document
string_value:
--- end of page.page_number=13 ---

## **Prompt for Integrating Part 1** 

You are the IntegrateAgent. Your job is to build an integrated factual summary for a QUESTION. YOU ARE GIVEN: - QUESTION: what must be answered. - EVIDENCE_CONTEXT: newly retrieved supporting evidence that may contain facts relevant to the QUESTION. - RESULT: the current working notes / draft summary about this same QUESTION (may be incomplete). 

## YOUR OBJECTIVE: 

Produce an UPDATED_RESULT that is a consolidated factual summary of all information that is relevant to the QUESTION. This is NOT a final answer to the QUESTION. It is an integrated summary of all useful facts that could be used to answer the QUESTION. 

The UPDATED_RESULT must: 1. Keep useful, correct, on-topic information from RESULT. 2. Add any new, relevant, well-supported facts from EVIDENCE_CONTEXT. 3. Remove anything that is off-topic for the QUESTION.


key: chroma:document
string_value:
INSTRUCTIONS: 1. Understand the QUESTION. Identify exactly what needs to be answered. 2. From RESULT: - Keep any statements that are relevant to the QUESTION. 3. From EVIDENCE_CONTEXT: - Extract every fact that helps describe, clarify, or support an answer to the QUESTION. - Prefer concrete details such as entities, numbers, versions, decisions, timelines, outcomes, responsibilities, constraints. - Ignore anything unrelated to the QUESTION. 4. Synthesis: - Merge the selected content from RESULT with the selected content from EVIDENCE_CONTEXT. - The merged text MUST read as one coherent factual summary related to the QUESTION (not the direct answer). - The merged summary MUST collect all important factual information needed to answer the QUESTION, so it can stand alone later without needing RESULT or EVIDENCE_CONTEXT. - Do NOT add interpretation, recommendations, or conclusions beyond what is explicitly stated in RESULT or EVIDENCE_CONTEXT. 

--- end of page.page_number=14 ---


key: chroma:document
string_value:
**Prompt for Integrating Part 2** RULES: - "content" MUST ONLY include factual information that is relevant to the QUESTION. - You are NOT producing a final answer, decision, recommendation, or plan. You are producing a cleaned, merged factual summary. - Do NOT invent or infer facts that do not appear in RESULT or EVIDENCE_CONTEXT. - Do NOT include meta language (e.g. "the evidence says", "according to RESULT", "the model stated"). - Do NOT include instructions, reasoning steps, or analysis of your own process. - Do NOT include any keys other than "content" and "sources". - "sources" should on incluede the page_ids of the pages that supported the included facts. THINKING STEP - Before producing the output, think about selection and synthesis steps inside <think>...</think>. - Keep the <think> concise but sufficient to ensure correctness and relevance. - After </think>, output ONLY the JSON object. The <think> section must NOT be included in the JSON. OUTPUT JSON SPEC: Return ONE JSON


key: chroma:document
string_value:
concise but sufficient to ensure correctness and relevance. - After </think>, output ONLY the JSON object. The <think> section must NOT be included in the JSON. OUTPUT JSON SPEC: Return ONE JSON object with EXACTLY: - "content": string. This is the UPDATED_RESULT, i.e. the integrated final information related to the QUESTION, if there not exist any useful information, just provide "". - "sources": array of strings/objects. Both keys MUST be present. After the <think> section, return ONLY the JSON object. Do NOT output Markdown, comments, headings, or explanations outside the JSON.


key: chroma:document
string_value:
**Prompt for InfoCheck** You are the InfoCheckAgent. Your job is to judge whether the currently collected information is sufficient to answer a specific QUESTION. YOU ARE GIVEN: - REQUEST: the QUESTION that needs to be answered. - RESULT: the current integrated factual summary about that QUESTION. RESULT is intended to contain all useful known information so far. YOUR OBJECTIVE: Decide whether RESULT already contains all of the information needed to fully answer REQUEST with specific, concrete details. You are NOT answering REQUEST. You are only judging completeness. **REQUEST** : {request} **RESULT** : {result} EVALUATION PROCEDURE: 1. Decompose REQUEST: - Identify the key pieces of information that are required to answer REQUEST completely (facts, entities, steps, reasoning, comparisons, constraints, timelines, outcomes, etc.). 2. Check RESULT: - For each required piece, check whether RESULT already provides that information clearly and specifically. - RESULT must be specific enough


key: chroma:document
string_value:
constraints, timelines, outcomes, etc.). 2. Check RESULT: - For each required piece, check whether RESULT already provides that information clearly and specifically. - RESULT must be specific enough that someone could now write a final answer directly from it without needing further retrieval. 3. Decide completeness: - "enough" = true  ONLY IF RESULT covers all required pieces with sufficient clarity and specificity. - "enough" = false otherwise. THINKING STEP - Before producing the output, perform your decomposition and evaluation inside <think>...</think>. - Keep the <think> concise but ensure it verifies completeness rigorously. - After </think>, output ONLY the JSON object with the key specified below. The <think> section must NOT be included in the JSON. OUTPUT REQUIREMENTS: Return ONE JSON object with EXACTLY this key: - "enough": boolean. true if RESULT is sufficient to answer REQUEST fully; false otherwise. RULES: - Do NOT invent facts. - Do NOT answer REQUEST. - Do NOT


key: chroma:document
string_value:
ONE JSON object with EXACTLY this key: - "enough": boolean. true if RESULT is sufficient to answer REQUEST fully; false otherwise. RULES: - Do NOT invent facts. - Do NOT answer REQUEST. - Do NOT include any explanation, reasoning, or extra keys. - After the <think> section, return ONLY the JSON object.


key: chroma:document
string_value:
--- end of page.page_number=16 ---

## **Prompt for Requests Generating** 

You are the FollowUpRequestAgent. Your job is to propose targeted follow-up retrieval questions for missing information. 

## YOU ARE GIVEN: 

- REQUEST: the original QUESTION that we ultimately want to be able to answer. - RESULT: the current integrated factual summary about this QUESTION. RESULT represents everything we know so far. 

## YOUR OBJECTIVE: 

Identify what important information is still missing from RESULT in order to fully answer REQUEST, and generate focused retrieval questions that would fill those gaps. 

## **REQUEST** : {request} 

**RESULT** : {result} 

## INSTRUCTIONS: 

1. Read REQUEST and determine what information is required to answer it completely (facts, numbers, definitions, procedures, timelines, responsibilities, comparisons, outcomes, constraints, etc.). 

2. Read RESULT and determine which of those required pieces are still missing, unclear, or underspecified.


key: chroma:document
string_value:
2. Read RESULT and determine which of those required pieces are still missing, unclear, or underspecified. 

3. For each missing piece, generate ONE standalone retrieval question that would directly obtain that missing information. 

- Each question MUST: 

- mention concrete entities / modules / components / datasets / events if they are known, - ask for factual information that could realistically be found by retrieval (not "analyze", "think", "infer", or "judge"). 

4. Rank the questions from most critical missing information to least critical. 

5. Produce at most 5 questions. 

## THINKING STEP 

- Before producing the output, reason about gaps and prioritize inside <think>...</think>. 

- Keep the <think> concise but ensure prioritization makes sense. - After </think>, output ONLY the JSON object specified below. The <think> section must NOT be included in the JSON. 

## OUTPUT FORMAT: 

Return ONE JSON object with EXACTLY this key:


key: chroma:document
string_value:
## OUTPUT FORMAT: 

Return ONE JSON object with EXACTLY this key: 

- "new_requests": array of strings (0 to 5 items). Each string is one retrieval question. 

## RULES: 

- Do NOT include any extra keys besides "new_requests". 

- After the <think> section, do NOT include explanations, reasoning steps, or Markdown outside the JSON. 

- Do NOT generate vague requests like "Get more info". 

- Do NOT answer REQUEST yourself. 

- Do NOT invent facts that are not asked by REQUEST. 

After the <think> section, return ONLY the JSON object. 

--- end of page.page_number=17 ---


key: chroma:document
string_value:
## **SAM 3D: 3Dfy Anything in Images** 

**SAM 3D Team** , **Xingyu Chen** _[∗]_ , **Fu-Jen Chu** _[∗]_ , **Pierre Gleize** _[∗]_ , **Kevin J Liang** _[∗]_ , **Alexander Sax** _[∗]_ , **Hao Tang** _[∗]_ , **Weiyao Wang** _[∗]_ , **Michelle Guo** , **Thibaut Hardin** , **Xiang Li** _[◦]_ , **Aohan Lin** , **Jiawei Liu** , **Ziqi Ma** _[◦]_ , **Anushka Sagar** , **Bowen Song** _[◦]_ , **Xiaodong Wang** , **Jianing Yang** _[◦]_ , **Bowen Zhang** _[◦]_ , **Piotr Dollár** _[†]_ , **Georgia Gkioxari** _[†]_ , **Matt Feiszli** _[†§]_ , **Jitendra Malik** _[†§]_ 

Meta Superintelligence Labs 

> _∗_ Core Contributor (Alphabetical, Equal Contribution), _◦_ Intern, _†_ Project Lead, _§_ Equal Contribution


key: chroma:document
string_value:
We present SAM 3D, a generative model for visually grounded 3D object reconstruction, predicting geometry, texture, and layout from a single image. SAM 3D excels in natural images, where occlusion and scene clutter are common and visual recognition cues from context play a larger role. We achieve this with a human- and model-in-the-loop pipeline for annotating object shape, texture, and pose, providing visually grounded 3D reconstruction data at unprecedented scale. We learn from this data in a modern, multi-stage training framework that combines synthetic pretraining with real-world alignment, breaking the 3D “data barrier”. We obtain significant gains over recent work, with at least a 5 : 1 win rate in human preference tests on real-world objects and scenes. We will release our code and model weights, an online demo, and a new challenging benchmark for in-the-wild 3D object reconstruction.


key: chroma:document
string_value:
**Demo:** https://www.aidemos.meta.com/segment-anything/editor/convert-image-to-3d **Code:** https://github.com/facebookresearch/sam-3d-objects **Website:** https://ai.meta.com/sam3d 

**==> picture [5 x 6] intentionally omitted <==**

**==> picture [4 x 6] intentionally omitted <==**

**==> picture [472 x 135] intentionally omitted <==**

**Figure 1 SAM 3D converts a single image into a composable 3D scene made of individual objects.** Our method predicts per-object geometry, texture, and layout, enabling full scene reconstruction. Bottom: high-quality 3D assets recovered for each object. 

## **1 Introduction**


key: chroma:document
string_value:
## **1 Introduction** 

In this paper (see Figure 1) we present SAM 3D, a generative neural network for 3D reconstruction from a single image. The model can reconstruct 3D shape and texture for any object, as well as its layout with respect to the camera, even in complex scenes with significant clutter and occlusion. As the reconstruction is of full 3D shape, not just of the visible 2.5D surface, one can then re-render the object from any desired viewpoint. 

Computer vision has traditionally focused on multi-view geometry as providing the primary signal for 3D shape. However psychologists (and artists before them) have long known that _humans_ can perceive depth and 

--- end of page.page_number=1 ---


key: chroma:document
string_value:
shape from a single image, _e.g_ . Koenderink et al. (1992) demonstrated this elegantly by showing that humans can estimate surface normals at probe points on an object’s image, which can then be integrated to a full surface. In psychology textbooks these single image cues to 3D shape are called “pictorial cues”, and include information such as in shading and texture patterns, but also recognition - the “familiar object” cue. In computer vision, this line of research dates back to Roberts (1963), who showed that once an image pattern was recognized as a known object, its 3D shape and pose could be recovered. The central insight is that recognition enables 3D reconstruction, an idea that has since resurfaced in different technical instantiations (Debevec et al., 2023; Cashman and Fitzgibbon, 2012; Kar et al., 2015; Gkioxari et al., 2019; Xiang et al., 2025). Note that this permits generalization to novel objects, because even if a specific object has not been seen before, it is made up


key: chroma:document
string_value:
2012; Kar et al., 2015; Gkioxari et al., 2019; Xiang et al., 2025). Note that this permits generalization to novel objects, because even if a specific object has not been seen before, it is made up of parts seen before.


key: chroma:document
string_value:
A fundamental challenge for learning such models is the lack of data: specifically, natural images paired with 3D ground truth are difficult to obtain at scale. Recent work (Yang et al., 2024b; Xiang et al., 2025) has shown strong reconstruction from single images. However, these models are trained on isolated objects and struggle with objects in natural scenes, where they may be distant or heavily occluded. To add such images to the training set, we need to find a way to associate specific objects in such images with 3D shape models, acknowledging that generalist human annotators find it hard to do so (unlike, say, attaching a label like “cat” or marking its boundary). Two insights made this possible: 

- We can create synthetic scenes where 3D object models are rendered and pasted into images (inspired by Dosovitskiy et al. (2015)).


key: chroma:document
string_value:
- We can create synthetic scenes where 3D object models are rendered and pasted into images (inspired by Dosovitskiy et al. (2015)). 

- While humans can’t easily _generate_ 3D shape models for objects, they can _select_ the likely best 3D model from a set of proffered choices and align its pose to the image (or declare that none of the choices is good).


key: chroma:document
string_value:
We design a training pipeline and data engine by adapting modern, multistage training recipes pioneered by LLMs (Minaee et al., 2025; Mo et al., 2025). As in recent works, we first train on a large collection of rendered synthetic objects. This is supervised pretraining: our model learns a rich vocabulary for object shape and texture, preparing it for real-world reconstruction. Next is mid-training with semi-synthetic data produced by pasting rendered models into natural images. Finally, post-training adapts the model to real images, using both a novel model-in-the-loop (MITL) pipeline and human 3D artists, and aligns it to human preference. We find that synthetic pretraining generalizes, given adequate post-training on natural images.


key: chroma:document
string_value:
Our post-training data, obtained from our MITL data pipeline, is key to obtaining good performance in natural images. Generalist human annotators aren’t capable of producing 3D shape ground truth; hence our annotators select and align 3D models to objects in images from the output of modules – computational and retrieval-based – that produce multiple initial 3D shape proposals. Human annotators select from these proposals, or route them to human artists for a subset of hard instances. The vetted annotations feed back into model training, and the improved model is reintegrated into the data engine to further boost annotation quality. This virtuous cycle steadily improves the quality of 3D annotations, labeling rates, and model performance.


key: chroma:document
string_value:
Due to the lack of prior benchmarks for real-world 3D reconstruction of object shape and layout, we propose a new evaluation set of 1 _,_ 000 image and 3D pairs: SAM 3D Artist Objects (SA-3DAO). The objects in our benchmark range from churches, ski lifts, and large structures to animals, everyday household items, and rare objects, and are paired with the real-world images in which they naturally appear. Professional 3D artists create 3D shapes from the input image, representing an expert human upper bound for visually grounded 3D reconstruction. We hope that contributing such an evaluation benchmark helps accelerate subsequent research iteration of real-world 3D reconstruction models. 

We summarize our contributions as follows:


key: chroma:document
string_value:
We summarize our contributions as follows: 

- We introduce **SAM 3D** , a new foundation model for 3D that predicts object shape, texture, and pose from a single image. By releasing code, model weights, and a demo, we hope to stimulate further advancements in 3D reconstruction and downstream applications of 3D. 

- We build a MITL pipeline for annotating shape, texture, and pose data, providing visually grounded 3D reconstruction data at unprecedented scale. 

--- end of page.page_number=2 ---

**==> picture [424 x 175] intentionally omitted <==**

**Figure 2 SAM 3D architecture.** ( **top** ) SAM 3D first predicts coarse shape and layout with the Geometry model; ( **right** ) the mixture of transformers architecture apply a two-stream approach with information sharing in the multi-modal self-attention layer. ( **bottom** ) The voxels predicted by the Geometry model are passed to the Texture & Refinement model, which adds higher resolution detail and textures.


key: chroma:document
string_value:
- We exploit this data via LLM-style pretraining and post-training in a novel framework for 3D reconstruction, combining synthetic pretraining with real-world alignment to overcome the orders of magnitude data gap between 3D and domains such as text, images, or video. 

- We release a challenging benchmark for real-world 3D object reconstruction, SA-3DAO. Experiments show SAM 3D’s significant gains via metrics and large-scale human preference. 

## **2 The SAM 3D Model** 

## **2.1 Problem Formulation**


key: chroma:document
string_value:
## **2 The SAM 3D Model** 

## **2.1 Problem Formulation** 

The act of taking a photograph maps a 3D object to a set of 2D pixels, specified by a mask _M_ in an image _I_ . We seek to invert this map. Let the object have shape _S_ , texture _T_ , and rotation, translation and scale ( _R, t, s_ ) in camera coordinates. Since the 3D to 2D map is lossy, we model the reconstruction problem as a conditional distribution _p_ ( _S, T, R, t, s|I, M_ ). Our goal is to train a generative model _q_ ( _S, T, R, t, s|I, M_ ) that approximates _p_ as closely as possible. 

## **2.2 Architecture** 

We build upon recent SOTA two-stage latent flow matching architectures (Xiang et al., 2025). SAM 3D first jointly predicts object pose and coarse shape, then refines the shapes by integrating pictorial cues (see Figure 2). Unlike Xiang et al. (2025) that reconstructs isolated objects, SAM 3D predicts object layout, creating coherent multi-object scenes.


key: chroma:document
string_value:
**Input encoding.** We use DINOv2 (Oquab et al., 2023) as an encoder to extract features from two pairs of images, resulting in 4 sets of conditioning tokens: 

- **Cropped object** : We encode the cropped image _I_ by mask _M_ and its corresponding _cropped binary mask_ , providing a focused, high-resolution view of the object. 

- **Full image** : We encode the full image _I_ and its _full image binary mask_ , providing global scene context and recognition cues absent from the cropped view. 

Optionally, the model supports conditioning on a coarse scene point map, _P_ obtained via hardware sensors ( _e.g_ ., LiDAR on an iPhone), or monocular depth estimation (Yang et al., 2024a; Wang et al., 2025a), enabling SAM 3D to integrate with other pipelines. 

--- end of page.page_number=3 ---

**==> picture [472 x 597] intentionally omitted <==**


key: chroma:document
string_value:
--- end of page.page_number=3 ---

**==> picture [472 x 597] intentionally omitted <==**

**Figure 3 SAM 3D data,** with a green outline around the target object, and the ground truth mesh shown in the bottom right. Samples are divided into four rows, based on type. Art-3DO meshes are untextured, while the rest may be textured or not, depending on the underlying asset (Iso-3DO, RP-3DO) or if the mesh was annotated for texture (MITL-3DO). 

--- end of page.page_number=4 ---

**==> picture [472 x 119] intentionally omitted <==**

**Figure 4 SAM 3D training paradigm.** We employ a multi-stage pipeline incrementally exposing the model to increasingly complex data and modalities.


key: chroma:document
string_value:
**Figure 4 SAM 3D training paradigm.** We employ a multi-stage pipeline incrementally exposing the model to increasingly complex data and modalities. 

**The Geometry Model** models the conditional distribution _p_ ( _O, R, t, s|I, M_ ), where _O ∈_ R[64][3] is coarse shape, _R ∈_ R[6] the 6D rotation (Zhou et al., 2019), _t ∈_ R[3] the translation, and _s ∈_ R[3] the scale. Conditioned on the input image and mask encodings, we employ a 1.2B parameter flow transformer with the Mixture-ofTransformers (MoT) architecture (Liang et al., 2025a; Deng et al., 2025), modeling geometry _O_ and layout ( _R, t, s_ ) using the attention mask in Figure 2. See Section C.1 for details.


key: chroma:document
string_value:
**The Texture & Refinement Model** learns the conditional distribution _p_ ( _S, T |I, M, O_ ). We first extract active voxels from the coarse shape _O_ predicted by Geometry model. A 600M parameter sparse latent flow transformer (Xiang et al., 2025; Peebles and Xie, 2023) refines geometric details and synthesizes object texture. 

**3D Decoders.** The latent representations from the Texture & Refinement Model can be decoded to either mesh or 3D Gaussian splats via a pair of VAE decoders _Dm_ , _Dg_ . These separately-trained decoders share the same VAE encoder and hence the same structured latent space (Xiang et al., 2025). We also detail several improvements in Section C.6. 

## **3 Training SAM 3D**


key: chroma:document
string_value:
## **3 Training SAM 3D** 

SAM 3D breaks the 3D data barrier using a recipe that progresses from synthetic pretraining to natural post-training, adapting the playbook from LLMs, robotics, and other large generative models. We build capabilities by stacking different training strategies in pre- and mid-training, and then align the model to real data and human-preferred behaviors through a post-training data flywheel. SAM 3D uses the following approach: 

**Step 1: Pretraining.** This phase builds foundational capabilities, such as shape generation, into a base model. 

**Step 1.5: Mid-Training.** Sometimes called continued pretraining, mid-training imparts general skills such as occlusion robustness, mask-following, and using visual cues.


key: chroma:document
string_value:
**Step 1.5: Mid-Training.** Sometimes called continued pretraining, mid-training imparts general skills such as occlusion robustness, mask-following, and using visual cues. 

**Step 2: Post-Training.** Post-training elicits target behavior, such as adapting the model from synthetic to real-world data or following human aesthetic preferences. We collect training samples ( _I, M_ ) _→_ ( _S, T, R, t, s_ ) and preference data from humans and use them in both supervised finetuning (SFT) and direct preference optimization (DPO) (Rafailov et al., 2023). 

This alignment (step 2) can be repeated, first collecting data with the current model and then improving the model with the new data. This creates a virtuous cycle with humans providing the supervision. Figure 10b shows that as we run the data engine longer, model performance steadily improves; dataset generation emerges as a byproduct of this alignment.


key: chroma:document
string_value:
The following sections detail the training objectives and data sources used in SAM 3D. We focus on the Geometry model; Texture & Refinement is trained similarly (details in Section C.5). Training hyper-parameters are in Section C.7. 

--- end of page.page_number=5 ---

|**Training stage**|**Modalities**|**Datasets**|**Condition input**|
|---|---|---|---|
|**Stage 1 Geometry model**||||
|Pre-training|_S, R_|Iso-3DO|object-centric crop|
|Mid-training|_S, R_<br>_S, R, t, s_|RP-3DO_†_<br>ProcThor, RP-3DO_‡_|full image<br>full image, pointmap_∗_|
|SFT|_S, R, t, s_|MITL, Art-3DO|full image, pointmap_∗_|
|Alignment|_S_|MITL preference|full image, pointmap_∗_|
|**Stage 2 Texture & Refinement**||**     model**||
|Pre-training|_T_|Iso-3DO-500K|object-centric crop|
|Mid-training|_T_|RP-3DO_§_|full image|
|SFT|_T_|MITL|full image|
|Alignment|_T_|MITL preference|full image|


key: chroma:document
string_value:
**Table 1 SAM 3D training stages.** _[†]_ Flying Occlusion (FO) from RP-3DO. _[‡]_ Object Swap - Random (OS-R) from RP-3DO. _[§]_ Object Swap - Annotated (OS-A) from RP-3DO. _[∗]_ optional. See Section B.2 for details. 

## **3.1 Pre & Mid-Training: Building a Base Model** 

Training begins with synthetic pretraining and mid-training, leveraging available large-scale datasets to learn strong priors for shape and texture, and skills such as mask-following, occlusion handling, and pose estimation. The rich features learned here drastically reduce the number of labeled real-world samples required in post-training (Hernandez et al., 2021), which generally incur acquisition costs. In pre- and mid-training, models are trained with rectified conditional flow matching (Liu et al., 2022) to generate multiple 3D modalities (see Section C.2). 

## **3.1.1 Pretraining: Single Isolated 3D Assets**


key: chroma:document
string_value:
## **3.1.1 Pretraining: Single Isolated 3D Assets** 

Pretraining trains the model to reconstruct accurate 3D shapes and textures from renders of isolated synthetic objects, following the successful recipes from (Xiang et al., 2025; Yang et al., 2024b; Wu et al., 2024). Specifically, we gather a set of image _I_ , shape _S_ , and texture _T_ triplets, using 2 _._ 7 million object meshes from Objaverse-XL (Deitke et al., 2023) and licensed datasets, and render them from 24 viewpoints, each producing a high-resolution image of a single centered object; more detail in Section B.1. We call this dataset _Iso-3DO_ and train for 2.5 trillion training tokens. 

## **3.1.2 Mid-Training: Semi-Synthetic Capabilities** 

Next, mid-training builds up foundational skills that will enable the model to handle objects in real-world images: 

- _Mask-following_ : We train the model to reconstruct a target object, defined by a binary mask on the input image.


key: chroma:document
string_value:
- _Mask-following_ : We train the model to reconstruct a target object, defined by a binary mask on the input image. 

- _Occlusion robustness_ : The artificial occluders in our dataset incentivize learning shape completion. 

- _Layout estimation_ : We train the model to produce translation and scale in normalized camera coordinates. 

We construct our data by rendering textured meshes into natural images using alpha compositing. This “render-paste” dataset contains one subset of occluder-occludee pairs, and another subset where we replace real objects with synthetic objects at similar location and scale, creating physically-plausible data with accurate 3D ground truth. We call this data _RP-3DO_ ; it contains 61 million samples with 2 _._ 8 million unique meshes; Figure 3 shows examples. See Section B.2 for more details.


key: chroma:document
string_value:
After mid-training (2.7 trillion training tokens), the model has now been trained with all input and output modalities for visually grounded 3D reconstruction. However, all data used has been (semi-)synthetic; to both close the domain gap and fully leverage real-world cues, we need real images. 

--- end of page.page_number=6 ---

**==> picture [472 x 100] intentionally omitted <==**

**----- Start of picture text -----**<br>
STAGE 1 STAGE 2 STAGE 2.5 STAGE 3<br>Goal of the stage<br>Methodology<br>Stages Sampler Shape Modeling<br>< <<br>3D Artists<br>3D Shape Rotation Translation Scale<br>Inaccurate<br>Object Selection
 3D Shape /<br>& Masking Texture Preference Accurate Alignment to 2.5D Scene<br>Human + SAM Human + Model Suite Human + Point Cloud<br>Image<br>**----- End of picture text -----**<br>


key: chroma:document
string_value:
**Figure 5 Life of an example going through the data collection pipeline.** We streamline annotation by breaking it into subtasks: annotators first choose target objects (Stage 1); rank and select 3D model candidates (Stage 2); then pose these models within a 2.5D scene (Stage 3). Stages 2 and 3 use model-in-the-loop. 

## **3.2 Post-Training: Real-World Alignment** 

In post-training, we have two goals. The first is to close the domain gap between (semi-)synthetic data and natural images. The second is to align with human preference for shape quality. We adapt the model by using our data engine iteratively; we first **(i) collect training data** with the current model, and then **(ii) update our model** using multi-stage post-training on this collected data. We then repeat. 

## **3.2.1 Post-Training: Collection Step**


key: chroma:document
string_value:
## **3.2.1 Post-Training: Collection Step** 

The core challenge with collecting data for 3D visual grounding is that most people cannot create meshes directly; this requires skilled 3D artists, who even then can take multiple hours. This is different from the segmentation masks collected in SAM (Kirillov et al., 2023). However, given options, most people _can_ choose which mesh resembles an object in the image most accurately. This fact forms the foundation of our data collection for SAM 3D. We convert preferences into training data as follows: sample from our post-trained model, ask annotators to choose the best candidate and then grade its overall quality according to a rubric which we define and update. If the quality meets the (evolving) bar, the candidate becomes a training sample.


key: chroma:document
string_value:
Unfortunately at the first iteration, our initial model yields few high-quality candidates. This is because before the first collection step, very little real-world data for 3D visual grounding exists. We deal with this cold start problem by leveraging a suite of existing learned and retrieval-based models to produce candidates. In early stages, we draw mostly from the ensemble, but as training progresses our best model dominates, eventually producing about 80% of the annotated data seen by SAM 3D.


key: chroma:document
string_value:
Our annotation pipeline collects 3D object shape _S_ , texture _T_ , orientation _R_ , 3D location _t_ , and scale _s_ from real-world images. We streamline the process by dividing into subtasks and leveraging existing appropriate models and human annotators within each (see Figure 5): identifying target objects, 3D model ranking and selection, and posing these within a 3D scene (relative to a point map). We outline each stage of the data engine below and present details in Section A. In total, we annotate almost 1 million images with _∼_ 3 _._ 14 million untextured meshes and _∼_ 100K textured meshes–unprecedented scale for 3D data paired with natural images.


key: chroma:document
string_value:
**Stage 1: Choosing target objects** ( _I, M_ ) **.** The goal of this stage is to identify a large, diverse set of images _I_ and object masks _M_ to lift to 3D. To ensure generalization across objects and scenes, we sample images from several diverse real-world datasets, and utilize a 3D-oriented taxonomy to balance the object distribution. To obtain object segmentation masks, we use a combination of pre-existing annotations (Kirillov et al., 2023) and human labelers selecting objects of interest. 

**Stage 2: Object model ranking and selection** ( _S, T_ ) **.** The goal of this stage is to collect image-grounded 3D shape _S_ and texture _T_ . As described above, human annotators choose shape and texture candidates which best match the input image and mask. Annotators rate the example _r_ and reject chosen examples that do not meet a predefined quality threshold, _i.e_ . _r < α_ . Bad candidates also become negative examples for preference alignment.


key: chroma:document
string_value:
Our data engine maximizes the likelihood of a successful annotation, _r > α_ , by asking annotators to choose 

--- end of page.page_number=7 ---

**==> picture [448 x 248] intentionally omitted <==**

**Figure 6 Qualitative comparison to competing image-to-3D asset methods.** We compare to the recent Trellis (Xiang et al., 2025), Hunyuan3D-2.1 (Hunyuan3D et al., 2025), Direct3D-S2 (Wu et al., 2025) and Hi3DGen (Ye et al., 2025) on the artist-generated SA-3DAO for single shape reconstruction; we provide the 3D artist-created ground truth mesh as reference. 

between _N_ = 8 candidates from the ensemble; a form of best-of- _N_ search (Ouyang et al., 2022) using humans. The expected quality of this best candidate improves with _N_ , and we further increase _N_ by first filtering using a model, and then filtering using the human (Anthony et al., 2017); we show results in Section A.7.


key: chroma:document
string_value:
**Stage 2.5: Hard example triage (Artists).** When no model produces a reasonable object shape, our non-specialist annotators cannot correct the meshes, resulting in a lack of data precisely where the model needs it most. We route a small percentage of these hardest cases to professional 3D artists for direct annotation, and we denote this set _Art-3DO_ . 

**Stage 3: Aligning objects to 2.5D scene** ( _R, t, s_ ) **.** The previous stages produce a 3D shape for the object, but not its layout in the scene. For each stage 2 shape, annotators label the object pose by manipulating the 3D object’s translation, rotation, and scale relative to a point cloud. We find that point clouds provide enough structure to enable consistent shape placement and orientation.


key: chroma:document
string_value:
In general, we can think of the data collection as an API that takes a current best model, _q_ ( _S, T, R, t, s | I, M_ ), and returns (i) training samples _D_[+] = ( _I, M, S, T, R, t, s_ ), (ii) a quality rating _r ∈_ [0 _,_ 1], and (iii) a set of less preferred candidates ( _D[−]_ = ( _I, M, S[′] , T[′] , R[′] , t[′] , s[′]_ )) that are all worse than the training sample. 

## **3.2.2 Post-Training: Model Improvement Step** 

The model improvement step in SAM 3D uses these training samples and preference results to update the base model through multiple stages of finetuning and preference alignment. Within each post-training iteration we aggregate data from all previous collection steps; keeping only samples where _D_[+] is above some quality threshold _α_ . As training progresses, _α_ can increase over time, similar to the cross-entropy method for optimization (de Boer et al., 2005). Our final post-training iteration uses 0.5 trillion training tokens.


key: chroma:document
string_value:
**Supervised Fine-Tuning (SFT).** When post-training begins, the base model has only seen synthetic data. Due to the large domain gap between synthetic and real-world data, we begin by finetuning on our aligned meshes from Stage 3. 

--- end of page.page_number=8 ---

**==> picture [448 x 248] intentionally omitted <==**

**Figure 7 Qualitative comparison to competing scene reconstruction methods.** We show SAM 3D’s full 3D scene reconstructions versus alternatives (Wen et al., 2024; Huang et al., 2025). 

We begin SFT with the noisier non-expert labels (MITL-3DO), followed by the smaller, high-quality set from 3D artists (Art-3DO). The high quality Art-3DO data enhances model quality by aligning outputs with artists’ aesthetic preferences. We find this helps suppress common failures, _e.g_ . floaters, bottomless meshes, and missing symmetry.


key: chroma:document
string_value:
**Preference optimization (alignment).** After fine-tuning, the model can robustly generate shape and layout for diverse objects and real-world images. However, humans are sensitive to properties like symmetry, closure, etc. which are difficult to capture with generic objectives like flow matching. Thus, we follow SFT with a stage of direct preference optimization (DPO) (Rafailov et al., 2023), using _D_ +/ _D[−]_ pairs from Stage 2 of our data engine. We found this off-policy data was effective at eliminating undesirable model outputs, even after SFT on Art-3DO. DPO training details are in Section C.3. 

**Distillation.** Finally, to enable sub-second shape and layout from the Geometry model, we finish a short distillation stage to reduce the number of function evaluations (NFE) required during inference from 25 _→_ 4. We adapt Frans et al. (2024) to our setting, and describe the details in Section C.4. 

## **4 Experiments**


key: chroma:document
string_value:
## **4 Experiments** 

**Dataset.** To comprehensively evaluate the model capability under real-world scenarios, we carefully build a new benchmark **SA-3DAO** , consisting of 1K 3D artist-created meshes created from natural images. We also include **ISO3D** from 3D Arena (Ebert, 2025) for quantitatively evaluating shape and texture, and Aria Digital Twin ( **ADT** ) (Pan et al., 2023) for layout. We further conduct human preference evaluation on two curated sets for both scene-level and object-level reconstruction. The **Pref Set** uses real-world images from MetaCLIP (Xu et al., 2024) and SA-1B (Kirillov et al., 2023), as well as a set based on LVIS (Gupta et al., 2019). Refer to Section D for details on evaluation sets.


key: chroma:document
string_value:
**Settings.** We conduct experiments with a Geometry model that is trained to condition on pointmaps. For datasets where pointmaps are unavailable, we estimate them with Wang et al. (2025a). We found that shape and texture quality do not depend on whether the model is trained with pointmap conditioning (see Section E.5), but layout (translation/scale) evaluation in Table 3 requires ground-truth depth/pointmap as 

--- end of page.page_number=9 ---


key: chroma:document
string_value:
--- end of page.page_number=9 ---

|Model|SA-3DAO<br>F1@0.01 (_↑_)<br>vIoU (_↑_)<br>Chamfer (_↓_)<br>EMD (_↓_)|ISO3D Eval Set<br>ULIP (_↑_)<br>Uni3D (_↑_)|
|---|---|---|
|Trellis<br>HY3D-2.1<br>HY3D-2.0<br>Direct3D-S2<br>TripoSG<br>Hi3DGen|0.1475<br>0.1392<br>0.0902<br>0.2131<br>0.1399<br>0.1266<br>0.1126<br>0.2432<br>0.1574<br>0.1504<br>0.0866<br>0.2049<br>0.1513<br>0.1465<br>0.0962<br>0.2160<br>0.1533<br>0.1445<br>0.0844<br>0.2057<br>0.1629<br>0.1531<br>0.0937<br>0.2134|0.1473<br>0.3698<br>0.1293<br>0.3546<br>0.1484<br>0.3662<br>0.1405<br>0.3653<br>**0.1529**<br>0.3687<br>0.1419<br>0.3594|
|**SAM 3D**|**0.2344**<br>**0.2311**<br>**0.0400**<br>**0.1211**|0.1488<br>**0.3707**|


key: chroma:document
string_value:
**Table 2 3D shape quantitative comparison** to competing image-to-3D methods, including Trellis (Xiang et al., 2025), HY3D-2.1 (Hunyuan3D et al., 2025), HY3D-2.0 (Team, 2025), Direct3D-S2 (Wu et al., 2025), TripoSG (Li et al., 2025), Hi3DGen (Ye et al., 2025). SA-3DAO shows metrics that measure accuracy against GT geometry; ISO3D (Ebert, 2025) has no geometric GT and so we show perceptual similarities between 3D and input images (ULIP (Xue et al., 2023) and Uni3D (Zhou et al., 2023)). TripoSG uses a significantly higher mesh resolution, which is rewarded in perceptual metrics. 

**==> picture [378 x 143] intentionally omitted <==**

**Figure 8 Preference comparison on scene-level and object-level reconstruction.** Numbers indicate human preference rates. Objects comparisons are done on textured meshes. SAM 3D is significantly preferred over others on all fronts. 

reference. 

## **4.1 Comparison with SOTA**


key: chroma:document
string_value:
reference. 

## **4.1 Comparison with SOTA** 

**3D shape and texture.** We evaluate single-object generation by comparing SAM 3D with prior state-of-the-art (SOTA) methods. In human preference studies, SAM 3D achieves an 5 : 1 head-to-head win rate on real images (see Figure 8). Table 2 presents quantitative evaluation on shape quality, where SAM 3D matches or exceeds previous SOTA performance on isolated object images ( **ISO3D** ), and significantly outperforms all baselines on challenging real-world inputs ( **SA-3DAO** ). Qualitative examples in Figure 6 further illustrate the model’s strong generalization under heavy occlusion. In Figure 9, we compare SAM 3D texture vs. other texture models, given SAM 3D shapes (SAM 3D’s improved shape actually benefits other methods in this eval). Annotators significantly prefer SAM 3D texture (details in Section E.2).


key: chroma:document
string_value:
**3D scene reconstruction.** In preference tests on three evaluation sets, users prefer scene reconstructions from SAM 3D by 6 : 1 over prior SOTA (Figure 8). Figure 7 and Figure 20 in the appendix shows qualitative comparisons, while Table 3 shows quantitative metrics for object layout. On real-world data like **SA-3DAO** and **ADT** , the improvement is fairly stark and persists even when _pipeline_ approaches use SAM 3D meshes. SAM 3D introduces a new real-world capability to generate shape and layout _jointly_ (ADD-S @ 0.1 2% _→_ 77%), and a sample-then-optimize approach, as in the render-and-compare approaches (Labbé et al., 2022; Wen et al., 2024) can further improve performance (Section E.3). The strong results for layout and scene reconstruction demonstrate that SAM 3D can robustly handle both RGB-only inputs ( _e.g_ ., **SA-3DAO** , **LVIS** , **Pref Set** ) as well 

--- end of page.page_number=10 ---


key: chroma:document
string_value:
|Generation<br>Model|SA-3DAO<br>3D IoU (_↑_)<br>ICP-Rot. (_↓_)<br>ADD-S (_↓_)<br>ADD-S @ 0.1 (_↑_)|Aria Digital Twin<br>3D IoU (_↑_)<br>ICP-Rot. (_↓_)<br>ADD-S (_↓_)<br>ADD-S @ 0.1 (_↑_)|
|---|---|---|
|Pipeline<br>Trellis + Megapose<br>Pipeline<br>HY3D-2.0 + Megapose<br>Pipeline<br>HY3D-2.0 + FoundationPose<br>Pipeline<br>HY3D-2.1 + FoundationPose<br>Pipeline<br>SAM 3D + FoundationPose<br>Joint<br>MIDI|0.2449<br>39.3866<br>0.5391<br>0.2831<br>0.2518<br>33.8307<br>0.7146<br>0.3647<br>0.2937<br>32.9444<br>0.3705<br>0.5396<br>0.2395<br>39.8357<br>0.4186<br>0.4177<br>0.2837<br>32.9168<br>0.3848<br>0.5079<br>-<br>-<br>-<br>-|0.2531<br>33.6114<br>0.4358<br>0.1971<br>0.3794<br>29.0066<br>0.1457<br>0.4211<br>0.3864<br>25.1435<br>0.1026<br>0.5992<br>0.2795<br>33.1197<br>0.2135<br>0.4129<br>0.3661<br>18.9102<br>0.0930<br>0.6495<br>0.0336<br>44.2353<br>2.5278<br>0.0175|


key: chroma:document
string_value:
|Joint<br>**SAM 3D**|**0.4254**<br>**20.7667**<br>**0.2661**<br>**0.7232**|**0.4970**<br>**15.2515**<br>**0.0765**<br>**0.7673**|


key: chroma:document
string_value:
**Table 3 3D layout quantitative comparison** to competing layout prediction methods on SA-3DAO and Aria Digital Twin (Pan et al., 2023). SAM 3D significantly outperforms both _pipeline_ approaches used in robotics (Labbé et al., 2022; Wen et al., 2024) and _joint_ generative models (MIDI (Huang et al., 2025)). Most SA-3DAO scenes only contain one object so we do not show MIDI results that require multi-object alignment. The metrics measure bounding box overlap, rotation error, and chamfer-like distances normalized by object diameter. 

**==> picture [330 x 124] intentionally omitted <==**

**Figure 9 Preference comparison on texture** . Since SAM 3D provides higher quality shape, we use the geometry results from SAM 3D and only perform texture generations for all methods. SAM 3D significantly outperforms others. 

as provided pointmaps ( _e.g_ ., **ADT** ). 

## **4.2 Analysis Studies**


key: chroma:document
string_value:
as provided pointmaps ( _e.g_ ., **ADT** ). 

## **4.2 Analysis Studies** 

**Post-training iterations steadily improve performance.** We observed steady improvements as we ran the data engine for longer, with near-linear Elo scaling shown in the historical comparisons from Stage 2 of our data engine (Figure 10a). We found it important to scale all stages simultaneously. The cumulatively linear effect results from more data engine iterations, along with scaling up pretraining, mid-training, and adding new post-training stages. Figure 10b shows that iterating MITL-3DO data alone yields consistent improvements but with decreasing marginal impact.


key: chroma:document
string_value:
**Multi-stage training improves performance.** SAM 3D’s real-world performance emerges through multi-stage training. Table 4 reveals near-monotonic 3D shape improvements as each training stage is added, validating the approach that leads to the final model (last row). In the appendix, Figure 17 shows similar results for texture and Table 7 shows the contribution of each individual real-world data stage, by knocking out the MITL-3DO, Art-3DO data, or DPO stages. 

**Other ablations.** Please see the appendix for additional ablations on rotation representation (Section E.4), DPO (Section C.3), distillation (Section C.4), pointmaps (Section E.5), and scaling best-of- _N_ in the data engine (Section A.7). 

## **5 Related Work**


key: chroma:document
string_value:
## **5 Related Work** 

**3D reconstruction** has been a longstanding challenge in computer vision. Classical methods include binocular stereopsis (Wheatstone, 1838), structure-from-motion (Hartley and Zisserman, 2003; Szeliski, 2022; Scharstein and Szeliski, 2002; Torresani et al., 2008; Tomasi and Kanade, 1992), and SLAM (Smith et al., 1990; Castellanos et al., 1999). Other strategies reconstruct by analysis (e.g., silhouettes (Esteban and Schmitt, 2004)) or by 

--- end of page.page_number=11 ---


key: chroma:document
string_value:
--- end of page.page_number=11 ---

|Training Stage|SA-3DAO<br>F1 @ 0.01 (_↑_)<br>vIoU (_↑_)<br>Chamfer (_↓_)<br>EMD (_↓_)|Preference set<br>Texture WR (_↑_)|
|---|---|---|
|Pre-training (Iso-3DO)<br>+ Mid-training (RP-3DO)<br>+ SFT (MITL-3DO)<br>+ DPO (MITL-3DO)<br>+ SFT (Art-3DO)<br>+ DPO (Art-3DO)|0.1349<br>0.1202<br>0.1036<br>0.2396<br>0.1705<br>0.1683<br>0.0760<br>0.1821<br>0.2027<br>0.2025<br>0.0578<br>0.1510<br>0.2156<br>0.2156<br>0.0498<br>0.1367<br>0.2331<br>**0.2337**<br>0.0445<br>0.1257<br>**0.2344**<br>0.2311<br>**0.0400**<br>**0.1211**|-<br>60.7<br>66.9<br>66.4<br>-<br>-|



**Table 4 Cascading improvements from multi-stage training on 3D shape and texture** . For texture, we report win rates (WR) between each row and the row _above_ it. 

**==> picture [189 x 163] intentionally omitted <==**

**==> picture [189 x 163] intentionally omitted <==**

**==> picture [372 x 11] intentionally omitted <==**


key: chroma:document
string_value:
**==> picture [189 x 163] intentionally omitted <==**

**==> picture [189 x 163] intentionally omitted <==**

**==> picture [372 x 11] intentionally omitted <==**

**----- Start of picture text -----**<br>
(a)  Historical Elo from data engine (b)  Impact of expanding training data<br>**----- End of picture text -----**<br>


**Figure 10 Data engine with additional iterations.** The plots show Elo scores of different models; a 400 point Elo difference corresponds to 10:1 odds in a preference test. Models were **(a)** checkpoints around 3 weeks apart, indicating cumulative improvements as we scale and add different stages and **(b)** post-trained (SFT) using expanded training data.


key: chroma:document
string_value:
synthesis via volume rendering (Kajiya and Von Herzen, 1984), using either implicit representations (Mildenhall et al., 2020) or explicit ones (Sitzmann et al., 2019; Liu et al., 2020). Supervised deep learning methods predict voxels (Xie et al., 2019; Wang et al., 2021), point clouds (Van Hoorick et al., 2022), or meshes (Worchel et al., 2022; Wen et al., 2019), or optimize implicit representations (Liu et al., 2024), e.g., signed distance functions (SDFs), often with high-quality output but requiring multiple views at inference. In contrast, we focus on the more restrictive setting of a single RGB image at test time.


key: chroma:document
string_value:
**Single-view 3D reconstruction** is considerably more difficult. A large body of work trains models with direct 3D supervision, predicting meshes (Xu et al., 2019; Kulkarni et al., 2022), voxels (Girdhar et al., 2016; Wu et al., 2017), point clouds (Fan et al., 2017; Mescheder et al., 2019), or CAD-aligned geometry (Wang et al., 2018; Gkioxari et al., 2019). A recent line of work (Zhang et al., 2023; Xiang et al., 2025; Ren et al., 2024) supervises with VAE (Kingma and Welling, 2013) latent representations. However, these methods are typically evaluated on simplified synthetic single-object benchmarks such as ShapeNet (Chang et al., 2015), Pix3D (Sun et al., 2018) or Objaverse (Deitke et al., 2023).


key: chroma:document
string_value:
**Layout estimation.** A large body of work estimates object poses from a single image, for object shapes (Labbé et al., 2022; Wen et al., 2024; Shi et al., 2025; Geng et al., 2025; Huang et al., 2025) or detections (Brazil et al., 2023), but is typically restricted to tabletop robotics, streets, or indoor scenes where objects rest on a supporting surface. In contrast, our approach estimates both pose for a broad range of object types across diverse scenes. 

**3D datasets.** Sourcing 3D annotations is challenging: the modality itself is complex, and the specialized tools required are hard to master. Anecdotally, modeling a 3D mesh from a reference image can take an 

--- end of page.page_number=12 ---


key: chroma:document
string_value:
--- end of page.page_number=12 ---

experienced artist hours (Section D). Instead, existing 3D datasets ( _e.g_ ., ShapeNet (Chang et al., 2015), Objaverse-XL (Deitke et al., 2023)) primarily consist of single synthetic objects; without paired real-world images, models can only learn from rendered views. In the real-world domain, existing datasets are small and mostly indoors (Reizenstein et al., 2021; Khanna et al., 2024; Fu et al., 2021; Szot et al., 2021; Pan et al., 2023). Models trained on such constrained data struggle to generalize.


key: chroma:document
string_value:
**Post-training.** While post-training began with a single supervised finetuning stage (Girshick et al., 2013; Wei et al., 2021), strong pretraining (Brown et al., 2020) made alignment much more data efficient (Hernandez et al., 2021), enabling iterative preference-based alignment like RLHF (Ouyang et al., 2022) and online DPO (Tang et al., 2024; Rafailov et al., 2023). When post-training must provide a strong steer, self-training methods offer denser supervision–leveraging the model itself to generate increasingly high-quality demonstrations, rather than relying solely on preference signals (Gulcehre et al., 2023; Anthony et al., 2017; Dong et al., 2023; Yuan et al., 2023). SAM 3D employs self-training to bridge the synthetic _→_ real domain gap and break the data barrier for 3D perception; most closely resembling RAFT (Dong et al., 2023), but also incorporating preference tuning.


key: chroma:document
string_value:
**Multi-stage pretraining.** Modern pretraining increasingly employs multiple training stages. Early work on curriculum learning (Bengio et al., 2009) provided a basis for staged data mixing in pretraining, with higherquality data coming later (Grattafiori et al., 2024; OLMo et al., 2025). Li et al. (2023b); Abdin et al. (2024) show that mixing synthetic/web curricula can achieve strong performance at smaller scales. Increasingly, additional mid-training stages are used for capability injection, such as context extension (Grattafiori et al., 2024) or coding (Rozière et al., 2024), and recent work finds that mid-training significantly improves posttraining effectiveness (Lambert, 2025; Wang et al., 2025b). SAM 3D introduces synthetic pretraining and mid-training that can generalize for 3D. 

## **6 Conclusion**


key: chroma:document
string_value:
## **6 Conclusion** 

We share SAM 3D: a new foundation model for full reconstruction of 3D shape, texture, and layout of objects from natural images. SAM 3D’s robustness on in-the-wild images, made possible by an innovative data engine and modern training recipe, represents a step change for 3D and an advance towards real-world 3D perception. With the release of our model, we expect SAM 3D to unlock new capabilities across diverse applications, such as robotics, AR/VR, gaming, film, and interactive media. 

## **Acknowledgements** 

We thank the following individuals for their contributions to this work:


key: chroma:document
string_value:
For their contributions to SAM Playground Engineering we thank: Robbie Adkins, Rene de la Fuente, Facundo Figueroa, Alex He, Dex Honsa, Alex Lende, Jonny Li, Peter Park, Don Pinkus, Roman Radle, Phillip Thomas, and Meng Wang. We thank our excellent XFN team for leadership and support: Kris Kitani, Vivian Lee, Sasha Mitts, George Orlin, Nikhila Ravi, and Andrew Westbury. Thanks to Helen Klein, Mallika Malhotra, and Azita Shokrpour for support with Legal, Privacy, and Integrity. We thank Michelle Chan, Kei Koyama, William Ngan, Yael Yungster for all the design support throughout the project. Thanks to Arpit Kalla for work on model efficiency. We thank Faye Ma and Kehan Lyu for data engineering support and tooling, and Emmanuel Hernandez, Robert Kuo for pipeline development. We thank Nan Yang for support with egocentric video data efforts. Thanks to our two interns Cem Gokmen, Jasmine Shone for their work on 3D and Lea Wilken for feedback on the manuscript. Thanks to our fantastic data


key: chroma:document
string_value:
for support with egocentric video data efforts. Thanks to our two interns Cem Gokmen, Jasmine Shone for their work on 3D and Lea Wilken for feedback on the manuscript. Thanks to our fantastic data operations team: Paris Baptiste, Karen Bergan, Kai Brown, Ida Cheng, Khadijat Durojaiye, Patrick Edwards, Daniella Factor, Eva Galper, Leonna Jones, Zayida Suber, Tatum Turner, Joseph Walker, and Claudette Ward.


key: chroma:document
string_value:
--- end of page.page_number=13 ---

## **References** 

- Marah Abdin et al. Phi-3 technical report: A highly capable language model locally on your phone, 2024. https: //arxiv.org/abs/2404.14219. 

- Thomas Anthony, Zheng Tian, and David Barber. Thinking fast and slow with deep learning and tree search. In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , NIPS’17, page 5366–5376, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964. 

- Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In _Proceedings of the 26th Annual International Conference on Machine Learning_ , ICML ’09, page 41–48, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585161. doi: 10.1145/1553374.1553380. https://doi.org/10. 1145/1553374.1553380.


key: chroma:document
string_value:
- Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discriminative components with random forests. In _European Conference on Computer Vision (ECCV)_ , 2014. 

- Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3d: A large benchmark and model for 3d object detection in the wild. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 13154–13164, 2023. 

- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. _Advances in neural information processing systems_ , 33:1877–1901, 2020. 

- Thomas J Cashman and Andrew W Fitzgibbon. What shape are dolphins? building 3d morphable models from 2d images. _IEEE transactions on pattern analysis and machine intelligence_ , 35(1):232–244, 2012.


key: chroma:document
string_value:
- Jose A Castellanos, José MM Montiel, José Neira, and Juan D Tardós. The SPmap: A probabilistic framework for simultaneous localization and map building. _IEEE Transactions on robotics and Automation_ , 1999. 

- Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. _arXiv preprint arXiv:1512.03012_ , 2015. 

- Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. The epic-kitchens dataset: Collection, challenges and baselines. _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 43(11):4125–4141, 2020.


key: chroma:document
string_value:
- Pieter-Tjerk de Boer, Dirk P. Kroese, Shie Mannor, and Reuven Y. Rubinstein. A tutorial on the cross-entropy method. _Annals of operations research_ , 134(1):19–67, January 2005. ISSN 0254-5330. doi: 10.1007/s10479-005-5724-z. 

- Paul E Debevec, Camillo J Taylor, and Jitendra Malik. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In _Seminal Graphics Papers: Pushing the Boundaries, Volume 2_ , 2023. 

- Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. _Advances in Neural Information Processing Systems_ , 36:35799–35813, 2023. 

- Yuxuan Deng, Yujia Zhu, Jiahui Chen, Yuan Wang, Yifei Li, Haotian Li, Junnan Li, Jinsheng Zhang, Wenhui Liu, Yuzheng Zhang, et al. Emerging properties in unified multimodal pretraining. _arXiv preprint arXiv:2505.14683_ , 2025.


key: chroma:document
string_value:
- Kristin Diehl and Cait Poynor. Great expectations?! assortment size, expectations, and satisfaction. _Journal of Marketing Research_ , 47(2):312–322, 2010. doi: 10.1509/jmkr.47.2.312. https://doi.org/10.1509/jmkr.47.2.312. 

- Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, KaShun SHUM, and Tong Zhang. RAFT: Reward ranked finetuning for generative foundation model alignment. _Transactions on Machine Learning Research_ , 2023. ISSN 2835-8856. https://openreview.net/forum?id=m7p5O7zblY. 

- Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. In _Proceedings of the IEEE international conference on computer vision_ , pages 2758–2766, 2015. 

Dylan Ebert. 3d arena: An open platform for generative 3d evaluation. _arXiv preprint arXiv:2506.18787_ , 2025.


key: chroma:document
string_value:
Dylan Ebert. 3d arena: An open platform for generative 3d evaluation. _arXiv preprint arXiv:2506.18787_ , 2025. 

--- end of page.page_number=14 ---

- Carlos Hernández Esteban and Francis Schmitt. Silhouette and stereo fusion for 3D object modeling. _Computer Vision and Image Understanding_ , 2004. 

- Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3D object reconstruction from a single image. In _CVPR_ , 2017. 

- Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. _arXiv preprint arXiv:2410.12557_ , 2024. 

- Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Binqiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 10933–10942, 2021.


key: chroma:document
string_value:
- Zheng Geng, Nan Wang, Shaocong Xu, Chongjie Ye, Bohan Li, Zhaoxi Chen, Sida Peng, and Hao Zhao. One view, many worlds: Single-image to 3d object meets generative domain randomization for one-shot 6d pose estimation. _arXiv preprint arXiv:2509.07978_ , 2025. 

- Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. Learning a predictable and generative vector representation for objects. In _ECCV_ , 2016. 

- Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. _CoRR_ , abs/1311.2524, 2013. http://arxiv.org/abs/1311.2524. 

- Georgia Gkioxari, Jitendra Malik, and Justin Johnson. Mesh r-cnn. In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 9785–9795, 2019. 

- Aaron Grattafiori et al. The llama 3 herd of models, 2024. https://arxiv.org/abs/2407.21783.


key: chroma:document
string_value:
- Aaron Grattafiori et al. The llama 3 herd of models, 2024. https://arxiv.org/abs/2407.21783. 

- Kristen Grauman, Andrew Westbury, et al. Ego4d: Around the world in 3,000 hours of egocentric video. _International Journal of Computer Vision (IJCV)_ , 2022. 

- Kristen Grauman et al. Ego-exo4d: Understanding skilled human activity from first- and third-person perspectives. _arXiv preprint arXiv:2401.10889_ , 2024. 

- Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, Wolfgang Macherey, Arnaud Doucet, Orhan Firat, and Nando de Freitas. Reinforced self-training (rest) for language modeling, 2023. https://arxiv.org/abs/2308.08998. 

- Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 5356–5364, 2019.


key: chroma:document
string_value:
Richard Hartley and Andrew Zisserman. Multiple view geometry in computer vision, 2003. 

- Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. _arXiv preprint arXiv:2102.01293_ , 2021. 

- Zehuan Huang, Yuan-Chen Guo, Xingqiao An, Yunhan Yang, Yangguang Li, Zi-Xin Zou, Ding Liang, Xihui Liu, Yan-Pei Cao, and Lu Sheng. Midi: Multi-instance diffusion for single image to 3d scene generation. In _Proceedings of the Computer Vision and Pattern Recognition Conference_ , pages 23646–23657, 2025. 

- Team Hunyuan3D, Shuhui Yang, Mingxin Yang, Yifei Feng, Xin Huang, Sheng Zhang, Zebin He, Di Luo, Haolin Liu, Yunfei Zhao, et al. Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material. _arXiv preprint arXiv:2506.15442_ , 2025. 

- James T Kajiya and Brian P Von Herzen. Ray tracing volume densities. _SIGGRAPH_ , 1984.


key: chroma:document
string_value:
- James T Kajiya and Brian P Von Herzen. Ray tracing volume densities. _SIGGRAPH_ , 1984. 

- Abhishek Kar, Shubham Tulsiani, Joao Carreira, and Jitendra Malik. Category-specific object reconstruction from a single image. In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1966–1974, 2015. 

- Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 16384–16393, 2024. 

- Diederik P Kingma and Max Welling. Auto-encoding variational bayes. _arXiv preprint arXiv:1312.6114_ , 2013. 

--- end of page.page_number=15 ---


key: chroma:document
string_value:
- Diederik P Kingma and Max Welling. Auto-encoding variational bayes. _arXiv preprint arXiv:1312.6114_ , 2013. 

--- end of page.page_number=15 ---

- Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 4015–4026, 2023. 

- Jan J Koenderink, Andrea J Van Doorn, and Astrid ML Kappers. Surface perception in pictures. _Perception & psychophysics_ , 52(5):487–496, 1992. 

- Nilesh Kulkarni, Justin Johnson, and David F. Fouhey. What’s behind the couch? directed ray distance functions for 3D scene reconstruction. In _ECCV_ , 2022.


key: chroma:document
string_value:
- Nilesh Kulkarni, Justin Johnson, and David F. Fouhey. What’s behind the couch? directed ray distance functions for 3D scene reconstruction. In _ECCV_ , 2022. 

- Yann Labbé, Lucas Manuelli, Arsalan Mousavian, Stephen Tyree, Stan Birchfield, Jonathan Tremblay, Justin Carpentier, Mathieu Aubry, Dieter Fox, and Josef Sivic. Megapose: 6d pose estimation of novel objects via render & compare. _arXiv preprint arXiv:2212.06870_ , 2022. 

Nathan Lambert. _Reinforcement Learning from Human Feedback_ . Online, 2025. https://rlhfbook.com. 

- Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language modeling, 2024. https://arxiv.org/abs/2403.13787.


key: chroma:document
string_value:
- Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models. _arXiv preprint arXiv:2502.06608_ , 2025. 

- Yanghao Li, Haoqi Fan, Rohit Girdhar, and Alexander Kirillov. Segment anything in videos. _arXiv preprint arXiv:2305.06500_ , 2023a. 

- Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report, 2023b. https://arxiv.org/abs/2309.05463. 

- Weixin Liang, LILI YU, Liang Luo, Srini Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen tau Yih, Luke Zettlemoyer, and Xi Victoria Lin. Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models. _Transactions on Machine Learning Research_ , 2025a. ISSN 2835-8856. https: //openreview.net/forum?id=Nu6N69i8SB.


key: chroma:document
string_value:
- Yixun Liang, Kunming Luo, Xiao Chen, Rui Chen, Hongyu Yan, Weiyu Li, Jiarui Liu, and Ping Tan. Unitex: Universal high fidelity generative texturing for 3d shapes. _arXiv preprint arXiv:2505.23253_ , 2025b. 

- Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. _NeurIPS_ , 2020. 

- Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10072–10083, 2024. 

- Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. _arXiv preprint arXiv:2209.03003_ , 2022.


key: chroma:document
string_value:
- Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. _arXiv preprint arXiv:2209.03003_ , 2022. 

- Zhaoyang Lv, Nicholas Charron, Pierre Moulon, Alexander Gamino, Cheng Peng, Chris Sweeney, Edward Miller, Huixuan Tang, Jeff Meissner, Jing Dong, et al. Aria everyday activities dataset. _arXiv preprint arXiv:2402.13349_ , 2024. 

- Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, Kevin Bailey, David Soriano Fosas, C. Karen Liu, Ziwei Liu, Jakob Engel, Renzo De Nardi, and Richard Newcombe. Nymeria: A massive collection of multimodal egocentric daily motion in the wild. In _the 18th European Conference on Computer Vision (ECCV)_ , 2024. https://arxiv.org/abs/2406.09905.


key: chroma:document
string_value:
- Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 4040–4048, 2016. 

- Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3D reconstruction in function space. In _CVPR_ , 2019. 

- Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In _ECCV_ , 2020. 

--- end of page.page_number=16 ---

- Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey, 2025. https://arxiv.org/abs/2402.06196.


key: chroma:document
string_value:
- Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey, 2025. https://arxiv.org/abs/2402.06196. 

- Kaixiang Mo, Yuxin Shi, Weiwei Weng, Zhiqiang Zhou, Shuman Liu, Haibo Zhang, and Anxiang Zeng. Mid-training of large language models: A survey, 2025. https://arxiv.org/abs/2510.06826.


key: chroma:document
string_value:
- Kaixiang Mo, Yuxin Shi, Weiwei Weng, Zhiqiang Zhou, Shuman Liu, Haibo Zhang, and Anxiang Zeng. Mid-training of large language models: A survey, 2025. https://arxiv.org/abs/2510.06826. 

- NVIDIA, :, Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang, Jiannan Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. Gr00t n1: An open foundation model for generalist humanoid robots. _arXiv preprint arXiv: 2503.14734_ , 2025.


key: chroma:document
string_value:
- Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Allyson Ettinger, Michal Guerquin, David Heineman, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Jake Poznanski, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. 2 olmo 2 furious, 2025. https://arxiv.org/abs/2501.00656. 

- Abby O’Neill et al. Open x-embodiment: Robotic learning datasets and rt-x models. In _2024 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 6892–6903. IEEE, 2024.


key: chroma:document
string_value:
- Abby O’Neill et al. Open x-embodiment: Robotic learning datasets and rt-x models. In _2024 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 6892–6903. IEEE, 2024. 

- Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. _arXiv preprint arXiv:2304.07193_ , 2023. 

- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. _arXiv preprint arXiv: 2203.02155_ , 2022.


key: chroma:document
string_value:
- Xiaqing Pan, Nicholas Charron, Yongqian Yang, Scott Peters, Thomas Whelan, Chen Kong, Omkar Parkhi, Richard Newcombe, and Yuheng Carl Ren. Aria digital twin: A new benchmark dataset for egocentric 3d machine perception. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 20133–20143, 2023. 

- William Peebles and Saining Xie. Scalable diffusion models with transformers. In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 4195–4205, 2023. 

- Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. _Advances in neural information processing systems_ , 36:53728–53741, 2023.


key: chroma:document
string_value:
- Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 10901–10911, 2021. 

- Xuanchi Ren, Jiahui Huang, Xiaohui Zeng, Ken Museth, Sanja Fidler, and Francis Williams. Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 4209–4219, 2024. 

- Lawrence G Roberts. _Machine perception of three-dimensional solids_ . PhD thesis, Massachusetts Institute of Technology, 1963.


key: chroma:document
string_value:
- Lawrence G Roberts. _Machine perception of three-dimensional solids_ . PhD thesis, Massachusetts Institute of Technology, 1963. 

- Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code, 2024. https://arxiv.org/abs/2308.12950. 

- Daniel Scharstein and Richard Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. _IJCV_ , 2002. 

- Wubin Shi, Shaoyan Gai, Feipeng Da, and Zeyu Cai. Sampose: Generalizable model-free 6d object pose estimation via single-view prompt. _IEEE Robotics and Automation Letters_ , 2025. 

--- end of page.page_number=17 ---


key: chroma:document
string_value:
--- end of page.page_number=17 ---

- Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, and Michael Zollhofer. DeepVoxels: Learning persistent 3D feature embeddings. In _CVPR_ , 2019. 

- Randall Smith, Matthew Self, and Peter Cheeseman. Estimating uncertain spatial relationships in robotics. In _Autonomous robot vehicles_ , 1990. 

- Julian Straub, Daniel DeTone, Tianwei Shen, Nan Yang, Chris Sweeney, and Richard Newcombe. Efm3d: A benchmark for measuring progress towards 3d egocentric foundation models. In _arXiv preprint arXiv:2406.10224_ , 2024. https://arxiv.org/abs/2406.10224. 

- Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman. Pix3d: Dataset and methods for single-image 3d shape modeling. In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2974–2983, 2018.


key: chroma:document
string_value:
Richard Szeliski. _Computer vision: algorithms and applications_ . Springer Nature, 2022. 

- Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. Habitat 2.0: Training home assistants to rearrange their habitat. _Advances in neural information processing systems_ , 34:251–266, 2021. 

- Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov, Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, and Will Dabney. Understanding the performance gap between online and offline alignment algorithms. _arXiv preprint arXiv: 2405.08448_ , 2024. 

- Tencent Hunyuan3D Team. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation, 2025. 

- Carlo Tomasi and Takeo Kanade. Shape and motion from image streams under orthography: a factorization method. _IJCV_ , 1992.


key: chroma:document
string_value:
- Carlo Tomasi and Takeo Kanade. Shape and motion from image streams under orthography: a factorization method. _IJCV_ , 1992. 

- Lorenzo Torresani, Aaron Hertzmann, and Chris Bregler. Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. _PAMI_ , 2008. 

- Basile Van Hoorick, Purva Tendulkar, Dídac Surís, Dennis Park, Simon Stent, and Carl Vondrick. Revealing occlusions with 4d neural fields. In _CVPR_ , 2022. 

- Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In _CVPR_ , 2024. 

- Dan Wang, Xinrui Cui, Xun Chen, Zhengxia Zou, Tianyang Shi, Septimiu Salcudean, Z Jane Wang, and Rabab Ward. Multi-view 3d reconstruction with transformers. In _ICCV_ , 2021.


key: chroma:document
string_value:
- Dan Wang, Xinrui Cui, Xun Chen, Zhengxia Zou, Tianyang Shi, Septimiu Salcudean, Z Jane Wang, and Rabab Ward. Multi-view 3d reconstruction with transformers. In _ICCV_ , 2021. 

- Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2Mesh: Generating 3D mesh models from single RGB images. In _ECCV_ , 2018. 

- Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In _Proceedings of the Computer Vision and Pattern Recognition Conference_ , pages 5261–5271, 2025a. 

- Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu. Octothinker: Mid-training incentivizes reinforcement learning scaling, 2025b. https://arxiv.org/abs/2506.20512.


key: chroma:document
string_value:
- Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu. Octothinker: Mid-training incentivizes reinforcement learning scaling, 2025b. https://arxiv.org/abs/2506.20512. 

- Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. _CoRR_ , abs/2109.01652, 2021. https: //arxiv.org/abs/2109.01652. 

- Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 17868–17879, 2024. 

- Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. Pixel2Mesh++: Multi-view 3D mesh generation via deformation. In _ICCV_ , 2019.


key: chroma:document
string_value:
- Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. Pixel2Mesh++: Multi-view 3D mesh generation via deformation. In _ICCV_ , 2019. 

- Charles Wheatstone. Contributions to the physiology of vision.—part the first. on some remarkable, and hitherto unobserved, phenomena of binocular vision. _Philosophical transactions of the Royal Society of London_ , 1838. 

- Markus Worchel, Rodrigo Diaz, Weiwen Hu, Oliver Schreer, Ingo Feldmann, and Peter Eisert. Multi-view mesh reconstruction with neural deferred shading. In _CVPR_ , 2022. 

--- end of page.page_number=18 ---

- Jiajun Wu, Yifan Wang, Tianfan Xue, Xingyuan Sun, Bill Freeman, and Josh Tenenbaum. MarrNet: 3D shape reconstruction via 2.5D sketches. _NeurIPS_ , 2017. 

- Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Jingxi Xu, Philip Torr, Xun Cao, and Yao Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. _Advances in Neural Information Processing Systems_ , 37: 121859–121881, 2024.


key: chroma:document
string_value:
- Shuang Wu, Youtian Lin, Feihu Zhang, Yifei Zeng, Yikang Yang, Yajie Bao, Jiachen Qian, Siyu Zhu, Philip Torr, Xun Cao, and Yao Yao. Direct3d-s2: Gigascale 3d generation made easy with spatial sparse attention. _arXiv preprint arXiv:2505.17412_ , 2025. 

- Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. In _Proceedings of the Computer Vision and Pattern Recognition Conference_ , pages 21469–21480, 2025. 

- Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. In _Robotics: Science and Systems (RSS)_ , 2018. 

- Haozhe Xie, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhou, and Shengping Zhang. Pix2vox: Context-aware 3D reconstruction from single and multi-view images. In _ICCV_ , 2019.


key: chroma:document
string_value:
- Haozhe Xie, Hongxun Yao, Xiaoshuai Sun, Shangchen Zhou, and Shengping Zhang. Pix2vox: Context-aware 3D reconstruction from single and multi-view images. In _ICCV_ , 2019. 

- Hu Xu, Nikhila Goyal, Mitchell Wortsman, Gabriel Ilharco, Ozan Sener, Aniruddha Kembhavi, Ali Farhadi, and Rohit Girdhar. Metaclip: How to make clip efficiently. _arXiv preprint arXiv:2404.07143_ , 2024. 

- Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, and Ulrich Neumann. DISN: Deep implicit surface network for high-quality single-view 3D reconstruction. _NeurIPS_ , 2019. 

- Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 1179–1189, 2023.


