394 lines
7.5 KiB
Markdown
394 lines
7.5 KiB
Markdown
# MoMentry Playground - AGENTS.md
|
||
|
||
## 概述
|
||
|
||
MoMentry Playground 是一個整合式桌面應用程式，提供：
|
||
- 多格式媒體播放（視頻、音頻、圖片）
|
||
- Frame-精確的視頻控制
|
||
- ASR 字幕顯示（可開關）
|
||
- YOLO 檢測框疊加（可開關）
|
||
- Chunk 導航標記（可開關）
|
||
- 與 momentry_core 處理模組的進度監控
|
||
- 自然語言搜尋
|
||
|
||
## 技術棧
|
||
|
||
| 元件 | 技術 |
|
||
|------|------|
|
||
| 語言 | Rust 2021 |
|
||
| 桌面框架 | tao + wry |
|
||
| 視頻播放 | SDL2 + FFmpeg |
|
||
| 前端 | HTML/CSS/JS (WebView) |
|
||
| 後端接口 | HTTP API (momentry_core port 3002) |
|
||
| 數據庫 | PostgreSQL, MongoDB, Qdrant (via API) |
|
||
| 緩存 | LRU (YOLO frame cache) |
|
||
|
||
---
|
||
|
||
## 構建/測試/代碼質量命令
|
||
|
||
### 構建
|
||
```bash
|
||
# Debug 構建
|
||
cargo build
|
||
|
||
# Release 構建
|
||
cargo build --release
|
||
|
||
# 運行
|
||
cargo run -- --video "/path/to/video.mov"
|
||
|
||
# 指定參數
|
||
cargo run --release -- \
|
||
--video "/Users/accusys/test_video/Old_Time_Movie_Show_-_Charade_1963.HD.mov" \
|
||
--asr "/Users/accusys/momentry_core_0.1/output/39567a0eb16f39fd.asr.json" \
|
||
--yolo "/Users/accusys/test_video/Old_Time_Movie_Show_-_Charade_1963.HD.yolo.json"
|
||
```
|
||
|
||
### 測試
|
||
```bash
|
||
# 運行所有測試
|
||
cargo test
|
||
|
||
# 運行單個測試
|
||
cargo test test_name
|
||
|
||
# 運行並顯示輸出
|
||
cargo test -- --nocapture
|
||
|
||
# 運行 doc tests
|
||
cargo test --doc
|
||
```
|
||
|
||
### 代碼質量
|
||
```bash
|
||
# 格式化代碼
|
||
cargo fmt
|
||
|
||
# 檢查格式化
|
||
cargo fmt -- --check
|
||
|
||
# Lint + 自動修復
|
||
cargo clippy --fix --allow-dirty --allow-staged
|
||
|
||
# Lint 檢查
|
||
cargo clippy
|
||
|
||
# 類型檢查
|
||
cargo check
|
||
|
||
# 完整檢查
|
||
cargo build --all-targets
|
||
```
|
||
|
||
---
|
||
|
||
## 代碼風格指南
|
||
|
||
### 格式規範
|
||
- **最大行長**: 100 字符
|
||
- **縮進**: 4 空格（使用 `rustfmt` 自動處理）
|
||
|
||
### 命名慣例
|
||
```rust
|
||
// 類型/結構體/枚舉: PascalCase
|
||
struct VideoPlayer;
|
||
enum PlaybackState;
|
||
trait MediaViewer;
|
||
|
||
// 函數/方法/變量: snake_case
|
||
fn play_video();
|
||
let current_frame = 0;
|
||
|
||
// 常量: SCREAMING_SNAKE_CASE
|
||
const MAX_FRAME_BUFFER: usize = 10;
|
||
```
|
||
|
||
### 錯誤處理
|
||
```rust
|
||
use anyhow::{Context, Result, anyhow};
|
||
|
||
fn load_video(path: &str) -> Result<VideoFile> {
|
||
let file = File::open(path)
|
||
.with_context(|| format!("Failed to open video: {}", path))?;
|
||
Ok(file)
|
||
}
|
||
|
||
#[derive(Debug, thiserror::Error)]
|
||
pub enum PlayerError {
|
||
#[error("Codec not supported: {0}")]
|
||
UnsupportedCodec(String),
|
||
|
||
#[error("Seek failed: {0}")]
|
||
SeekFailed(String),
|
||
}
|
||
```
|
||
|
||
### 模塊結構
|
||
```rust
|
||
// src/lib.rs
|
||
pub mod player;
|
||
pub mod overlay;
|
||
pub mod web;
|
||
pub mod config;
|
||
|
||
// src/player/mod.rs
|
||
pub mod video;
|
||
pub mod ffmpeg;
|
||
pub mod renderer;
|
||
pub mod state;
|
||
|
||
pub use video::Video;
|
||
pub use state::{PlayerState, PlaybackState};
|
||
```
|
||
|
||
---
|
||
|
||
## 快捷鍵定義
|
||
|
||
| 快捷鍵 | 功能 | Toggle |
|
||
|--------|------|--------|
|
||
| `Space` | 播放/暫停 | - |
|
||
| `←` | 上一幀 | - |
|
||
| `→` | 下一幀 | - |
|
||
| `Shift+←` | 後退 1 秒 | - |
|
||
| `Shift+→` | 前進 1 秒 | - |
|
||
| `S` | 字幕顯示 | ✅ |
|
||
| `Y` | YOLO 疊加 | ✅ |
|
||
| `C` | Chunk 標記 | ✅ |
|
||
| `M` | 靜音 | ✅ |
|
||
| `+` / `=` | 放大 | - |
|
||
| `-` | 縮小 | - |
|
||
| `0` | 重置縮放 | - |
|
||
| `R` | 重置視圖 | - |
|
||
| `F` | 全屏 | - |
|
||
| `[` | 上一 Chunk | - |
|
||
| `]` | 下一 Chunk | - |
|
||
| `/` | 開啟搜尋 | - |
|
||
| `Esc` | 關閉面板 | - |
|
||
|
||
---
|
||
|
||
## 命令列參數
|
||
|
||
```bash
|
||
momentry [OPTIONS]
|
||
|
||
Options:
|
||
-v, --video <PATH> Video file path
|
||
-a, --asr <PATH> ASR JSON file path
|
||
-y, --yolo <PATH> YOLO JSON file path
|
||
-w, --width <N> Window width (default: 1280)
|
||
-h, --height <N> Window height (default: 720)
|
||
--fullscreen Start in fullscreen mode
|
||
--locale <LANG> UI language (en, zh-TW)
|
||
--help Show help
|
||
```
|
||
|
||
---
|
||
|
||
## 數據格式規範
|
||
|
||
### ASR JSON
|
||
```json
|
||
{
|
||
"language": "en",
|
||
"language_probability": 0.99,
|
||
"segments": [
|
||
{ "start": 0.0, "end": 19.04, "text": "Hello and welcome..." }
|
||
]
|
||
}
|
||
```
|
||
|
||
### YOLO JSON
|
||
```json
|
||
{
|
||
"metadata": { "fps": 59.94, "total_frames": 412343, "width": 1920, "height": 1080 },
|
||
"frames": {
|
||
"100": {
|
||
"frame_number": 100,
|
||
"time_seconds": 1.668,
|
||
"detections": [
|
||
{ "class_name": "person", "confidence": 0.85, "x1": 150, "y1": 200, "x2": 400, "y2": 800 }
|
||
]
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
**座標系**: 絕對像素座標 (0 ~ width/height)
|
||
|
||
---
|
||
|
||
## 服務連接配置
|
||
|
||
```yaml
|
||
PostgreSQL:
|
||
host: localhost
|
||
port: 5432
|
||
database: momentry
|
||
user: accusys
|
||
password: Test3200
|
||
|
||
Redis:
|
||
host: localhost
|
||
port: 6379
|
||
password: accusys
|
||
|
||
Qdrant:
|
||
host: localhost
|
||
port: 6333
|
||
collection: AccusysDB
|
||
api_key: Test3200Test3200Test3200
|
||
|
||
API Server:
|
||
host: 127.0.0.1
|
||
port: 3002
|
||
|
||
Ollama:
|
||
host: localhost
|
||
port: 11434
|
||
```
|
||
|
||
---
|
||
|
||
## 測試數據
|
||
|
||
```bash
|
||
VIDEO="/Users/accusys/test_video/Old_Time_Movie_Show_-_Charade_1963.HD.mov"
|
||
ASR="/Users/accusys/momentry_core_0.1/output/39567a0eb16f39fd.asr.json"
|
||
YOLO="/Users/accusys/test_video/Old_Time_Movie_Show_-_Charade_1963.HD.yolo.json"
|
||
```
|
||
|
||
---
|
||
|
||
## 項目結構
|
||
|
||
```
|
||
momentry_playground/
|
||
├── Cargo.toml
|
||
├── src/
|
||
│ ├── main.rs # 入口
|
||
│ ├── lib.rs # 模組導出
|
||
│ ├── config.rs # 命令列解析
|
||
│ ├── player/ # 視頻播放
|
||
│ │ ├── mod.rs
|
||
│ │ ├── video.rs # 播放控制
|
||
│ │ ├── ffmpeg.rs # FFmpeg 解碼
|
||
│ │ ├── renderer.rs # SDL2 渲染
|
||
│ │ └── state.rs # 播放狀態
|
||
│ ├── overlay/ # 疊加層
|
||
│ │ ├── mod.rs
|
||
│ │ ├── asr.rs # ASR 載入
|
||
│ │ └── yolo.rs # YOLO 載入 (LRU cache)
|
||
│ └── web/ # WebView
|
||
│ ├── mod.rs
|
||
│ └── bridge.rs # JS <-> Rust IPC
|
||
├── web/
|
||
│ ├── index.html # 主 UI
|
||
│ ├── styles.css # 樣式
|
||
│ └── app.js # 前端邏輯
|
||
├── AGENTS.md
|
||
└── README.md
|
||
```
|
||
|
||
---
|
||
|
||
## 外部依賴
|
||
|
||
### FFmpeg (必需)
|
||
```bash
|
||
# macOS
|
||
brew install ffmpeg
|
||
|
||
# Linux
|
||
sudo apt install ffmpeg
|
||
|
||
# 驗證
|
||
ffmpeg -version
|
||
ffprobe -version
|
||
```
|
||
|
||
### yt-dlp (YouTube 支援)
|
||
```bash
|
||
brew install yt-dlp
|
||
pip install yt-dlp
|
||
```
|
||
|
||
---
|
||
|
||
## 開發工作流
|
||
|
||
### 1. 創建功能分支
|
||
```bash
|
||
git checkout -b feature/video-controls
|
||
```
|
||
|
||
### 2. 開發與測試
|
||
```bash
|
||
cargo test feature_name
|
||
cargo clippy --fix
|
||
cargo fmt
|
||
cargo build --release
|
||
```
|
||
|
||
### 3. 提交
|
||
```bash
|
||
git commit -m "feat(player): add frame-precise seeking"
|
||
git commit -m "fix(controls): correct volume sync"
|
||
```
|
||
|
||
### 4. 推送與 PR
|
||
```bash
|
||
git push -u origin feature/video-controls
|
||
# 在 Gitea 創建 PR → main
|
||
```
|
||
|
||
---
|
||
|
||
## 常見問題
|
||
|
||
### Q: FFmpeg 路徑找不到
|
||
在 `src/player/ffmpeg.rs` 中指定路徑：
|
||
```rust
|
||
Command::new("/opt/homebrew/bin/ffmpeg")
|
||
```
|
||
|
||
### Q: YOLO 檔案太大 (483MB)
|
||
使用 LRU cache 緩存幀：
|
||
```rust
|
||
const CACHE_SIZE: usize = 60; // 只緩存前後 30 幀
|
||
cache: LruCache<u64, Vec<Detection>>
|
||
```
|
||
|
||
### Q: 編譯緩慢
|
||
```bash
|
||
cargo install sccache
|
||
export RUSTC_WRAPPER=sccache
|
||
```
|
||
|
||
---
|
||
|
||
## 多語言支援 (i18n)
|
||
|
||
目前界面語言為 English。擴展時：
|
||
1. 在 `web/` 添加 `locales/` 目錄
|
||
2. 建立 `en.json`, `zh-TW.json` 等檔案
|
||
3. 在 `app.js` 載入對應語言檔案
|
||
4. 使用 `data-i18n` 屬性標記翻譯元素
|
||
|
||
---
|
||
|
||
## 實施優先順序
|
||
|
||
| Phase | 功能 | 優先級 |
|
||
|-------|------|--------|
|
||
| 1 | 基礎播放器 | P0 |
|
||
| 2 | ASR 字幕 | P1 |
|
||
| 3 | YOLO 疊加 | P1 |
|
||
| 4 | Chunk 標記 | P2 |
|
||
| 5 | Zoom/Pan | P1 |
|
||
| 6 | Vector 搜尋 | P2 |
|
||
| 7 | 進度監控 | P3 |
|
||
| 8 | 雲端/YouTube | P3 |
|