readme update

tigerlcl · tigerlcl · commit 5710282a1b19 · 2025-09-10T15:35:35.000+08:00
diff --git a/README.md b/README.md
@@ -1,41 +1,67 @@
-# ArxivFlow
-Workflow - Periodic Track on Arxiv.org Paper
+# ArxivFlow - Periodic Track on arXiv Paper
+
+English | [中文](README_CN.md)
 
 Author: Tiger, member from [HKUST Dial](https://github.com/HKUSTDial)
 
 Last update: September 09, 2025
 
-# Objectives
-This workflow serves for tracking daily updates in Arxiv.org. Paper info will be preprocessed and concluded by a series of modules. Finally, it will post to a group chat in Feishu for reading. The target audience is for education and research community.
+## 🎯 Objectives
+This workflow serves for tracking daily updates in arXiv.org. Paper info will be preprocessed and concluded by a series of modules. Finally, it will post to a group chat in Feishu for reading. The target audience is for education and research community.
+
+> 💰 Cost: less than 0.05 CNY per workflow execution.
+
+## ✨ Key Features
+
+- 📚 Automatically fetch latest arXiv papers
+- 🤖 AI-powered paper summarization and filtering
+- 📱 Auto-send to Feishu group chat
+- ⏰ GitHub Actions automated scheduling
+- 🛠️ Local debugging script support
+
+## 📋 Prerequisites
+
+Before getting started, please ensure you have prepared the following accounts and services:
+
+1. **[Dify](https://dify.ai/) account** - Free registration for building AI workflows
+2. **LLM Provider API** - Recommended [DeepSeek API](https://platform.deepseek.com/api_keys) (cost-effective)
+3. **[Jina](https://jina.ai/) API key** - For web content extraction, new users get 1M free credits
+4. **Feishu Group Bot Webhook** - For message pushing
+
+## 🚀 Quick Start
+
+### Step 1: Setup Dify Workflow
+
+1. **Open Dify Console**
+   - Login to Dify and find the "Studio" tab
+   ![](image/dify_studio.png)
 
-> Cost: less than 0.05 CNY per workflow execution.
+2. **Import Workflow**
+   - Create a new workflow by importing [this DSL file](dsl/ArxivDairy.yml)
+   - This DSL file contains the complete logic for paper fetching, processing, and pushing
 
-# Prerequisites
-- A [Dify](https://dify.ai/) account, sign up for free plan.
-- A LLM provider API (I use [DeepSeek API](https://platform.deepseek.com/api_keys))
-- A [Jina](https://jina.ai/) API key (1M Free credits for new users)
-- A Group Bot webhook (For Feishu group chat robot)
+3. **Configure Environment Variables**
+   - Configure necessary environment variables in workflow settings
+   ![](image/env.png)
+   - See detailed configuration in **Environment Variables Configuration** section below
 
-# How to build your own workflow?
+4. **Get API Token**
+   - Get your workflow API token from workflow settings
+   - This token will be used for automated scheduling
 
-## Step 1: Setup Dify Workflow
-1. Open your Dify Cloud and find the "Studio" tab.![](image/dify_studio.png)
-2. Create a new workflow via import [this](dsl/ArxivDairy.yml) DSL file
-3. Configure the environment variables in the workflow. ![](image/env.png)
-4. Get your workflow API token from the workflow settings
+### Step 2: Setup Automated Scheduler (Recommended)
 
-## Step 2: Setup Automated Scheduler (Recommended)
-The project now includes an integrated scheduler that runs your workflow automatically. Since your Dify workflow already handles Feishu messaging internally, the scheduler simply executes the workflow and logs the results.
+The project provides an integrated scheduler that can trigger Dify-side workflows on schedule.
 
-### Quick Setup:
-1. **Configure GitHub Secrets**: Go to your repository Settings > Secrets and variables > Actions > New repository secret, and add:
-   - `DIFY_TOKENS`: Your Dify workflow API token(s) - separate multiple tokens with `;`
-   - `DIFY_BASE_URL`: (Optional) Dify API URL, defaults to `https://api.dify.ai/v1`
-   - `DIFY_INPUTS`: (Optional) JSON format input variables if your workflow requires them
+#### Quick Setup:
 
-2. **Enable GitHub Actions**: Go to the Actions tab in your repository and enable workflows
+1. **Configure GitHub Secrets**:
+   - Go to repository Settings > Secrets and variables > Actions > New repository secret
+   - Add secret `DIFY_TOKENS`: Your Dify workflow API token (separate multiple tokens with `;`)
 
-3. **Automatic Execution**: The scheduler runs daily at 06:30 Beijing Time automatically
+2. **Enable GitHub Actions**: Go to repository Actions tab and enable workflows
+
+3. **Automatic Execution**: The scheduler will automatically run according to timing rules defined in [dify-scheduler.yml](.github/workflows/dify-scheduler.yml). For syntax details, see [cron.help](https://cron.help/).
 
 ### Manual Execution:
 - **GitHub Actions**: Go to Actions tab > "Dify ArxivFlow Scheduler" > "Run workflow"
@@ -47,7 +73,8 @@ The project now includes an integrated scheduler that runs your workflow automat
   npm start
   ```
 
-## Final Result
+### 📱 Final Result
+
 ![](image/feishu_demo.png)
 
 The scheduler will automatically:
@@ -56,29 +83,47 @@ The scheduler will automatically:
 - ❌ Report any errors to GitHub Actions logs
 - 🔄 Support multiple workflows if needed
 
-# Scheduler Configuration
-
-## Environment Variables
-Configure these as GitHub repository secrets or local environment variables:
+## 🔧 Environment Variables Configuration
 
-### Required Variables:
-- `DIFY_TOKENS`: Your Dify workflow API token(s). For multiple workflows, separate with `;`
+### GitHub Actions Secrets (Required):
+- `DIFY_TOKENS`: Your Dify workflow API token, separate multiple workflows with `;`
 
-### Optional Variables:
+### Optional Configuration:
 - `DIFY_BASE_URL`: Dify API base URL (default: `https://api.dify.ai/v1`)
-- `DIFY_INPUTS`: JSON format input variables for workflows (default: `{}`)
+- `DIFY_INPUTS`: Workflow input variables in JSON format (default: `{}`)
+
+### Dify Workflow Internal Environment Variables:
+- `FEISHU_DEV` / `FEISHU_PROD`: Feishu Group Bot Webhook for testing/production environments
+- `JINA`: API key for crawling arXiv search results
+- `KEYWORDS`: Keywords for arXiv paper search, comma-separated
+  - The number of KEYWORDS and sending frequency needs to match the timing rules in GitHub Actions
+  - Example: If sending 4 pushes daily, KEYWORDS needs 4 keywords, and timing rules need 4 time points
+- `PAPER_NUM_MAX`: Maximum number of papers per message (limited by Feishu message length)
+
+## 🛠️ Debugging Scripts
 
-## Original Dify Workflow Env Vars:
-- `FEISHU_DEV` / `FEISHU_PROD`: Webhook of Feishu Group Bot for testing/deployment
-- `JINA`: Web crawler API key for Arxiv.org
-- `KEYWORDS`: Comma-separated keywords for Arxiv query
-- `PAPER_NUM_MAX`: Maximum number of papers per message (Feishu message limits)
+The `/scripts` folder contains scripts for local debugging and testing, simulating the processes used in Dify Workflow:
 
+- **`jina_extract.py`**: Simulates Jina API calls and paper information extraction logic
+- **`sample.text`**: Sample data returned by Jina API for local testing
+- **`extracted_papers.json`**: Example of structured paper data after extraction, serves as input for downstream LLM analysis in workflow
 
-# Acknowledgement
+These scripts help you test and debug paper extraction logic without consuming API credits.
+
+### Usage for Local Development:
+```bash
+cd scripts
+python jina_extract.py
+```
+
+## 🤝 Acknowledgement
 - Dify Official Guidance: [Link](https://docs.dify.ai/docs/workflow/overview)
 - Feishu - How to use Bot in Group Chat: [Link (Chinese)](https://www.feishu.cn/hc/zh-CN/articles/360024984973-%E5%9C%A8%E7%BE%A4%E7%BB%84%E4%B8%AD%E4%BD%BF%E7%94%A8%E6%9C%BA%E5%99%A8%E4%BA%BA?from=in-im-bot)
 - AWS Workshop: Lab3-使用Dify构建AI Workflow: [Link (Chinese)](https://catalog.us-east-1.prod.workshops.aws/workshops/2c19fcb1-1f1c-4f52-b759-0ca4d2ae2522/zh-CN)
 - arXiv Category: [Link](https://arxiv.org/category_taxonomy)
 - Dify Schedule Project: [Link](https://github.com/leochen-g/dify-schedule) - Inspiration for the automated scheduler implementation
 
+## 📄 License
+
+MIT License - See [LICENSE](LICENSE) file
+
diff --git a/README_CN.md b/README_CN.md
@@ -0,0 +1,134 @@
+# ArxivFlow - 定期追踪 arXiv论文更新
+
+[English](README.md) | 中文
+
+
+作者：Tiger，来自 [HKUST Dial](https://github.com/HKUSTDial) 实验室
+
+最后更新：2025年9月9日
+
+## 🎯 项目目标
+
+这个工作流用于追踪 arXiv.org 的日常论文更新。论文信息将通过一系列模块进行预处理和总结，最终发布到飞书群聊中供阅读。目标用户是教育和研究社区成员。
+
+> 💰 成本：每次工作流执行成本低于 0.05 人民币
+
+## ✨ 主要功能
+
+- 📚 自动抓取 arXiv 最新论文
+- 🤖 AI 智能总结和筛选论文
+- 📱 自动发送到飞书群聊
+- ⏰ GitHub Actions 自动定时执行
+- 🛠️ 本地调试脚本支持
+
+## 📋 前置条件
+
+在开始之前，请确保您已准备好以下账号和服务：
+
+1. **[Dify](https://dify.ai/) 账号** - 免费注册，用于构建 AI 工作流
+2. **LLM 提供商 API** - 推荐使用 [DeepSeek API](https://platform.deepseek.com/api_keys)（性价比高）
+3. **[Jina](https://jina.ai/) API 密钥** - 用于网页内容提取，新用户有100万免费额度
+4. **飞书群机器人 Webhook** - 用于消息推送
+
+## 🚀 快速开始
+
+### 第1步：设置 Dify 工作流
+
+1. **打开 Dify 控制台**
+   - 登录 Dify 并找到 "Studio" 选项卡
+   ![](image/dify_studio.png)
+
+2. **导入工作流**
+   - 创建新工作流，通过导入 [DSL 文件](dsl/ArxivDairy.yml) 
+   - 这个 DSL 文件包含了完整的论文抓取、处理和推送逻辑
+
+3. **配置环境变量**
+   - 在工作流设置中配置必要的环境变量
+   ![](image/env.png)
+   - 详细配置说明见下方**环境变量配置**部分
+
+4. **获取 API 令牌**
+   - 在工作流设置中获取您的工作流 API 令牌
+   - 这个令牌用于后续的自动化调度
+
+### 第2步：设置自动化调度器（推荐）
+
+项目提供一个集成的调度器，可以定时触发DiFy端侧工作流。
+
+#### 快速配置：
+
+1. **配置 GitHub Secrets**：
+   - 进入仓库的 Settings > Secrets and variables > Actions > New repository secret
+   - 添加密钥`DIFY_TOKENS`: 您的 Dify 工作流 API 令牌（多个令牌用 `;` 分隔）
+
+2. **启用 GitHub Actions**：进入仓库的 Actions 选项卡并启用工作流
+
+3. **自动执行**：调度器将依据 [dify-scheduler.yml](.github/workflows/dify-scheduler.yml) 中的定时规则自动运行，定义语法详见[cron.help](https://cron.help/)。
+
+#### 手动执行：
+
+- **GitHub Actions**：Actions 选项卡 > "Dify ArxivFlow Scheduler" > "Run workflow"
+- **本地测试**：
+  ```bash
+  npm install
+  # 设置环境变量
+  export DIFY_TOKENS="your_workflow_token_here"
+  npm start
+  ```
+
+## 🔧 环境变量配置
+
+### GitHub Actions 密钥（必需）：
+- `DIFY_TOKENS`: 您的 Dify 工作流 API 令牌，多个工作流用 `;` 分隔
+
+### 可选配置：
+- `DIFY_BASE_URL`: Dify API 基础 URL（默认：`https://api.dify.ai/v1`）
+- `DIFY_INPUTS`: 工作流输入变量，JSON 格式（默认：`{}`）
+
+### Dify 工作流内部环境变量：
+- `FEISHU_DEV` / `FEISHU_PROD`: 飞书群机器人 Webhook，用于测试/生产环境
+- `JINA`: 用于arXiv搜索结果的爬取API 密钥
+- `KEYWORDS`: 用于arXiv搜索论文关键词，逗号分隔
+   - KEYWORDS 数目和时间发送频率需要与GitHub Actions中的定时规则相匹配
+   - 例如：每天发送4次推送，那么KEYWORDS需要设置为4个关键词，定时规则也需要有4个时间点
+- `PAPER_NUM_MAX`: 每条消息的最大论文数量（受飞书消息长度限制）
+
+## 🛠️ 调试脚本说明
+
+`/scripts` 文件夹包含用于本地调试和测试的脚本，模拟 Dify 工作流中的处理过程：
+
+- **`jina_extract.py`**: 模拟 Jina API 调用和论文信息提取逻辑
+- **`sample.text`**: Jina API 返回的示例数据，用于本地测试
+- **`extracted_papers.json`**: 提取后的结构化论文数据示例，作为工作流后续大模型LLM分析的输入信息
+
+这些脚本帮助您在不消耗 API 额度的情况下测试和调试论文提取逻辑。
+
+### 本地调试使用方法：
+```bash
+cd scripts
+python jina_extract.py
+```
+
+## 📱 最终效果
+
+![](image/feishu_demo.png)
+
+调度器将自动：
+- ✅ 每日执行您的 Dify 工作流
+- 📊 记录执行结果和状态
+- ❌ 将任何错误报告到 GitHub Actions 日志
+- 🔄 根据需要支持多个工作流
+
+
+## 🤝 致谢
+
+- Dify 官方指南：[链接](https://docs.dify.ai/docs/workflow/overview)
+- 飞书 - 如何在群组中使用机器人：[链接](https://www.feishu.cn/hc/zh-CN/articles/360024984973-%E5%9C%A8%E7%BE%A4%E7%BB%84%E4%B8%AD%E4%BD%BF%E7%94%A8%E6%9C%BA%E5%99%A8%E4%BA%BA?from=in-im-bot)
+- AWS 工作坊：Lab3-使用Dify构建AI Workflow：[链接](https://catalog.us-east-1.prod.workshops.aws/workshops/2c19fcb1-1f1c-4f52-b759-0ca4d2ae2522/zh-CN)
+- arXiv 分类体系：[链接](https://arxiv.org/category_taxonomy)
+- Dify 调度项目：[链接](https://github.com/leochen-g/dify-schedule) - 自动化调度器实现的灵感来源
+
+## 📄 许可证
+
+MIT License - 详见 [LICENSE](LICENSE) 文件
+
