LucasAI111 commited on
Commit
142d4b2
·
verified ·
1 Parent(s): 0e2b30d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -17
README.md CHANGED
@@ -7,11 +7,11 @@ base_model:
7
  - Qwen/Qwen3-4B-Thinking-2507
8
  ---
9
 
10
- # DMind-2: Advanced Web3 Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation
11
 
12
  ## Model Overview
13
 
14
- DMind-2 is a series of Web3 investment analysis language models designed to provide real-time, professional Web3 investment consulting services for individual investors and professional institutions. Standing on the shoulders of numerous open-source pioneers, we have successfully launched two model variants through innovative post-training techniques. Among these, DMind-2-4B is specifically optimized for edge deployment, enabling users to access institutional-grade investment analysis capabilities on local devices without concerns about data privacy or network latency.
15
 
16
  ## Model Variants(DMind-2-4B)
17
 
@@ -26,9 +26,9 @@ DMind-2 is a series of Web3 investment analysis language models designed to prov
26
 
27
  ### 1. Domain-Adaptive Supervised Fine-Tuning (SFT)
28
 
29
- In building DMind-2, we deeply understand the uniqueness of the Web3 investment domain—it requires not only profound blockchain technical understanding but also keen financial market insights, and most importantly, the ability to perform rigorous logical reasoning among complex on-chain data and market signals. Therefore, our domain-adaptive fine-tuning strategy fully considers these requirements from the very beginning of dataset construction. We carefully curated a total of 47.6K high-quality training samples, including 27.8K Web3 domain-specific data points covering comprehensive Web3 investment scenarios from DeFi protocol analysis and NFT valuation models to DAO governance decisions. These data points are not simple Q&A pairs but contain complete investment logic chains, encompassing the entire reasoning process from market observation, data analysis, and risk assessment to investment recommendations.
30
 
31
- To ensure the model maintains fundamental financial analysis capabilities while focusing on the Web3 domain, we specifically incorporated 11.2K high-quality general domain data points and 8.6K pan-financial domain data points. These datasets help the model establish a solid foundation in financial theory and market analysis frameworks, enabling it to creatively apply mature methodologies from traditional finance to the emerging Web3 sector. Through this multi-layered data fusion strategy, DMind-2 can act like a professional investment advisor who understands both technology and finance, providing users with comprehensive and in-depth investment analysis.
32
 
33
  ### 2. 🔥 Core Innovation: Distribution-Preserving Chain-of-Thought Distillation (DPCD)
34
 
@@ -39,7 +39,7 @@ DMind-2's greatest technical breakthrough lies in our innovative Distribution-Pr
39
  The DPCD optimization objective combines domain adaptation with reasoning preservation through the following loss function:
40
 
41
  $$
42
- \mathcal{L}_{\text{DPCD}} = \underbrace{\mathcal{L}_{\text{CE}}(\theta_s, \mathcal{D}_{\text{Web3}})}_{\text{Domain Learning}} + \underbrace{\lambda(t) \cdot \sum_{i=1}^{T} \alpha_i \cdot D_{\text{KL}}(P_{\theta_s}^{(i)} \| P_{\theta_t}^{(i)})}_{\text{Distribution Preservation}} + \underbrace{\beta \cdot \mathcal{L}_{\text{QS}}(\mathcal{C}_{\theta_s})}_{\text{Quality Score}}
43
  $$
44
 
45
  Where:
@@ -60,13 +60,13 @@ $$
60
  \lambda(t) = \begin{cases}
61
  \lambda_{\text{high}} \cdot \left(1 + \tanh\left(\frac{\mathcal{H}(x_t) - \mu_{\mathcal{H}}}{\sigma_{\mathcal{H}}}\right)\right) & \text{if } \mathcal{T}(x_t) \in \{\text{DeFi Analysis, Risk Assessment}\} \\
62
  \lambda_{\text{base}} & \text{if } \mathcal{T}(x_t) \in \{\text{Market Data, Price Query}\} \\
63
- \lambda_{\text{base}} \cdot \left(1 + \frac{\mathcal{S}(c_t)}{|\mathcal{V}_{\text{Web3}}|}\right) & \text{otherwise}
64
  \end{cases}
65
  $$
66
 
67
- Where \\(\mathcal{H}(x_t)\\) measures reasoning complexity through chain length and branching factor, \\(\mathcal{S}(c_t)\\) counts domain-specific terms, and \\(|\mathcal{V}_{\text{Web3}}|\\) is the Web3 vocabulary size.
68
 
69
- This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep Web3 domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism \\(\mathcal{L}_{\text{QS}}\\) filters out low-quality reasoning chains, maintaining only those paths with coherence scores above threshold \\(\tau = 0.85\\).
70
 
71
  Through extensive experimentation, we found optimal hyperparameters: \\(\lambda_{\text{base}} = 0.3\\), \\(\lambda_{\text{high}} = 0.7\\), \\(\beta = 0.2\\), and \\(\delta = 0.1\\). This configuration achieves a 94.1% reasoning chain completeness while improving domain-specific accuracy by 23.2% over baseline fine-tuning methods.
72
 
@@ -76,7 +76,7 @@ After completing basic domain fine-tuning, we further optimize the model using t
76
 
77
  For professional output formatting, we constructed 4.2K carefully designed professional format data points. These data samples are sourced from real investment research reports, market analysis documents, and project due diligence reports, covering all aspects of investment analysis. Through RLHF training, the model learned how to organize a professional investment analysis report: starting with an executive summary that clearly articulates investment opportunities and risks; conducting in-depth technical analysis and market evaluation in the main body; and finally providing clear investment recommendations and risk warnings. This structured output not only improves information readability but more importantly helps investors establish systematic analytical frameworks, avoiding impulsive investment decisions due to disorganized information.
78
 
79
- Safety alignment is another aspect we particularly emphasize. The Web3 investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.
80
 
81
  ## Performance Metrics
82
 
@@ -92,23 +92,23 @@ Safety alignment is another aspect we particularly emphasize. The Web3 investmen
92
  | | CNMO 2024 (Pass@1) | 78.8 | 86.9 | - | - | 82.4 |
93
  | **Tools** | | | | | | |
94
  | | BFCL_v3 | - | 67.8 | 70.3 | 65.9 | 70.2 |
95
- | **Web3** | | | | | | |
96
  | | DMind Benchmark | 71.5 | 70.2 | 68.0 | 74.5 | 80.1 |
97
 
98
 
99
  ## Application Scenarios
100
 
101
- ### 🎯 Edge-Side Web3 Investment Decision Support
102
 
103
- DMind-2 can provide real-time Web3 investment analysis on users' personal devices, including DeFi yield comparisons, liquidity mining strategy optimization, and NFT valuation analysis. All calculations and analyses are completed locally, ensuring absolute privacy of investment strategies and position information. The model can analyze on-chain data, evaluate project fundamentals, identify market trends, and provide comprehensive support for investment decisions.
104
 
105
  ### 💼 Personalized Financial Advisory Services
106
 
107
- Based on users' risk preferences, investment objectives, and asset allocation needs, DMind-2 can provide customized investment advice. Whether for long-term value investing or short-term arbitrage opportunities, the model can provide professional analysis and recommendations. More importantly, it can explain complex Web3 concepts in plain language, helping investors understand the logic behind every investment decision.
108
 
109
  ### 📊 Comprehensive Financial Investment Computational Analysis
110
 
111
- DMind-2 is not limited to the Web3 domain but also possesses powerful pan-financial computational analysis capabilities. It can perform yield calculations, risk assessments, portfolio optimization, correlation analysis, and other professional financial computations. By integrating traditional financial theory with Web3 innovative mechanisms, the model helps investors find optimal asset allocation solutions between old and new financial systems.
112
 
113
  ### 🔍 Real-Time Market Monitoring and Alerts
114
 
@@ -160,13 +160,13 @@ print(response)
160
  ## Limitations & Disclaimers
161
 
162
  1. **Not Investment Advice**: Model outputs are for reference only; final investment decisions require users' own judgment
163
- 2. **Market Risk**: Web3 markets are highly volatile; please carefully assess risk tolerance
164
  3. **Knowledge Timeliness**: Model knowledge has temporal limitations; latest market information requires additional verification
165
  4. **Regulatory Compliance**: Please comply with financial regulations in your jurisdiction when using
166
 
167
  ## Acknowledgments
168
 
169
- We thank the Qwen and zai teams for providing the excellent base model and the continuous contributions from the open-source community. DMind-2's success wouldn't be possible without the collective efforts of the entire AI and Web3 community.
170
 
171
  ## License
172
 
@@ -176,7 +176,7 @@ This model follows the Apache 2.0 open-source license. Commercial use must compl
176
 
177
  ```bibtex
178
  @misc{dmind2025,
179
- title={DMind-2: Advanced Web3 Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation},
180
  author={DMind Team},
181
  year={2025},
182
  publisher={Hugging Face}
 
7
  - Qwen/Qwen3-4B-Thinking-2507
8
  ---
9
 
10
+ # DMind-2: Advanced Crypto Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation
11
 
12
  ## Model Overview
13
 
14
+ DMind-2 is a series of crypto investment analysis language models designed to provide real-time, professional crypto investment consulting services for individual investors and professional institutions. Standing on the shoulders of numerous open-source pioneers, we have successfully launched two model variants through innovative post-training techniques. Among these, DMind-2-4B is specifically optimized for edge deployment, enabling users to access institutional-grade investment analysis capabilities on local devices without concerns about data privacy or network latency.
15
 
16
  ## Model Variants(DMind-2-4B)
17
 
 
26
 
27
  ### 1. Domain-Adaptive Supervised Fine-Tuning (SFT)
28
 
29
+ In building DMind-2, we deeply understand the uniqueness of the crypto investment domain—it requires not only profound blockchain technical understanding but also keen financial market insights, and most importantly, the ability to perform rigorous logical reasoning among complex on-chain data and market signals. Therefore, our domain-adaptive fine-tuning strategy fully considers these requirements from the very beginning of dataset construction. We carefully curated a total of 47.6K high-quality training samples, including 27.8K crypto domain-specific data points covering comprehensive crypto investment scenarios from DeFi protocol analysis and NFT valuation models to DAO governance decisions. These data points are not simple Q&A pairs but contain complete investment logic chains, encompassing the entire reasoning process from market observation, data analysis, and risk assessment to investment recommendations.
30
 
31
+ To ensure the model maintains fundamental financial analysis capabilities while focusing on the crypto domain, we specifically incorporated 11.2K high-quality general domain data points and 8.6K pan-financial domain data points. These datasets help the model establish a solid foundation in financial theory and market analysis frameworks, enabling it to creatively apply mature methodologies from traditional finance to the emerging crypto sector. Through this multi-layered data fusion strategy, DMind-2 can act like a professional investment advisor who understands both technology and finance, providing users with comprehensive and in-depth investment analysis.
32
 
33
  ### 2. 🔥 Core Innovation: Distribution-Preserving Chain-of-Thought Distillation (DPCD)
34
 
 
39
  The DPCD optimization objective combines domain adaptation with reasoning preservation through the following loss function:
40
 
41
  $$
42
+ \mathcal{L}_{\text{DPCD}} = \underbrace{\mathcal{L}_{\text{CE}}(\theta_s, \mathcal{D}_{\text{crypto}})}_{\text{Domain Learning}} + \underbrace{\lambda(t) \cdot \sum_{i=1}^{T} \alpha_i \cdot D_{\text{KL}}(P_{\theta_s}^{(i)} \| P_{\theta_t}^{(i)})}_{\text{Distribution Preservation}} + \underbrace{\beta \cdot \mathcal{L}_{\text{QS}}(\mathcal{C}_{\theta_s})}_{\text{Quality Score}}
43
  $$
44
 
45
  Where:
 
60
  \lambda(t) = \begin{cases}
61
  \lambda_{\text{high}} \cdot \left(1 + \tanh\left(\frac{\mathcal{H}(x_t) - \mu_{\mathcal{H}}}{\sigma_{\mathcal{H}}}\right)\right) & \text{if } \mathcal{T}(x_t) \in \{\text{DeFi Analysis, Risk Assessment}\} \\
62
  \lambda_{\text{base}} & \text{if } \mathcal{T}(x_t) \in \{\text{Market Data, Price Query}\} \\
63
+ \lambda_{\text{base}} \cdot \left(1 + \frac{\mathcal{S}(c_t)}{|\mathcal{V}_{\text{crypto}}|}\right) & \text{otherwise}
64
  \end{cases}
65
  $$
66
 
67
+ Where \\(\mathcal{H}(x_t)\\) measures reasoning complexity through chain length and branching factor, \\(\mathcal{S}(c_t)\\) counts domain-specific terms, and \\(|\mathcal{V}_{\text{crypto}}|\\) is the crypto vocabulary size.
68
 
69
+ This mathematical framework ensures that DMind-2 maintains Qwen3's powerful reasoning capabilities while acquiring deep crypto domain expertise. The KL divergence constraint operates at each token generation step, preserving the original model's reasoning patterns. The quality scoring mechanism \\(\mathcal{L}_{\text{QS}}\\) filters out low-quality reasoning chains, maintaining only those paths with coherence scores above threshold \\(\tau = 0.85\\).
70
 
71
  Through extensive experimentation, we found optimal hyperparameters: \\(\lambda_{\text{base}} = 0.3\\), \\(\lambda_{\text{high}} = 0.7\\), \\(\beta = 0.2\\), and \\(\delta = 0.1\\). This configuration achieves a 94.1% reasoning chain completeness while improving domain-specific accuracy by 23.2% over baseline fine-tuning methods.
72
 
 
76
 
77
  For professional output formatting, we constructed 4.2K carefully designed professional format data points. These data samples are sourced from real investment research reports, market analysis documents, and project due diligence reports, covering all aspects of investment analysis. Through RLHF training, the model learned how to organize a professional investment analysis report: starting with an executive summary that clearly articulates investment opportunities and risks; conducting in-depth technical analysis and market evaluation in the main body; and finally providing clear investment recommendations and risk warnings. This structured output not only improves information readability but more importantly helps investors establish systematic analytical frameworks, avoiding impulsive investment decisions due to disorganized information.
78
 
79
+ Safety alignment is another aspect we particularly emphasize. The crypto investment field is full of high-risk, high-reward opportunities, and the model must accurately identify and highlight potential risks. We use proprietary risk case datasets to conduct safety training on the model, ensuring it won't output overly optimistic investment advice or overlook obvious risk signals. For example, when analyzing an emerging DeFi protocol, the model automatically checks key risk indicators such as smart contract audit status, team background, and total value locked, explicitly marking risk levels in investment recommendations. This responsible output approach not only protects users' asset security but also reflects our commitment to financial compliance.
80
 
81
  ## Performance Metrics
82
 
 
92
  | | CNMO 2024 (Pass@1) | 78.8 | 86.9 | - | - | 82.4 |
93
  | **Tools** | | | | | | |
94
  | | BFCL_v3 | - | 67.8 | 70.3 | 65.9 | 70.2 |
95
+ | **Crypto** | | | | | | |
96
  | | DMind Benchmark | 71.5 | 70.2 | 68.0 | 74.5 | 80.1 |
97
 
98
 
99
  ## Application Scenarios
100
 
101
+ ### 🎯 Edge-Side Crypto Investment Decision Support
102
 
103
+ DMind-2 can provide real-time crypto investment analysis on users' personal devices, including DeFi yield comparisons, liquidity mining strategy optimization, and NFT valuation analysis. All calculations and analyses are completed locally, ensuring absolute privacy of investment strategies and position information. The model can analyze on-chain data, evaluate project fundamentals, identify market trends, and provide comprehensive support for investment decisions.
104
 
105
  ### 💼 Personalized Financial Advisory Services
106
 
107
+ Based on users' risk preferences, investment objectives, and asset allocation needs, DMind-2 can provide customized investment advice. Whether for long-term value investing or short-term arbitrage opportunities, the model can provide professional analysis and recommendations. More importantly, it can explain complex crypto concepts in plain language, helping investors understand the logic behind every investment decision.
108
 
109
  ### 📊 Comprehensive Financial Investment Computational Analysis
110
 
111
+ DMind-2 is not limited to the crypto domain but also possesses powerful pan-financial computational analysis capabilities. It can perform yield calculations, risk assessments, portfolio optimization, correlation analysis, and other professional financial computations. By integrating traditional financial theory with crypto innovative mechanisms, the model helps investors find optimal asset allocation solutions between old and new financial systems.
112
 
113
  ### 🔍 Real-Time Market Monitoring and Alerts
114
 
 
160
  ## Limitations & Disclaimers
161
 
162
  1. **Not Investment Advice**: Model outputs are for reference only; final investment decisions require users' own judgment
163
+ 2. **Market Risk**: Crypto markets are highly volatile; please carefully assess risk tolerance
164
  3. **Knowledge Timeliness**: Model knowledge has temporal limitations; latest market information requires additional verification
165
  4. **Regulatory Compliance**: Please comply with financial regulations in your jurisdiction when using
166
 
167
  ## Acknowledgments
168
 
169
+ We thank the Qwen and zai teams for providing the excellent base model and the continuous contributions from the open-source community. DMind-2's success wouldn't be possible without the collective efforts of the entire AI and Crypto community.
170
 
171
  ## License
172
 
 
176
 
177
  ```bibtex
178
  @misc{dmind2025,
179
+ title={DMind-2: Advanced Crypto Domain-Specific Large Language Models with Distribution-Preserving CoT Distillation},
180
  author={DMind Team},
181
  year={2025},
182
  publisher={Hugging Face}