Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0a7242ddb3 | ||
|
|
d1f1112c65 | ||
|
|
dbe08e00a7 |
@@ -1,12 +1,61 @@
|
|||||||
# 自動注音 AutoZhuyin
|
# 自動注音 AutoZhuyin
|
||||||
|
|
||||||
macOS 繁體中文輸入法:**不用按切換鍵**,直接用大千注音鍵盤打字,輸入法會自動判斷你打的是注音還是英文。
|
給繁體中文使用者的輸入法(macOS、Windows):**不用按切換鍵**,直接用大千注音鍵盤打字,輸入法會自動判斷你打的是注音還是英文。
|
||||||
|
|
||||||
```
|
```
|
||||||
按鍵: ji3 hello world su3cl3␣
|
按鍵: ji3 hello world su3cl3␣
|
||||||
輸出: 我hello world 你好
|
輸出: 我hello world 你好
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 功能
|
||||||
|
|
||||||
|
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
|
||||||
|
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
|
||||||
|
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
|
||||||
|
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
|
||||||
|
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
|
||||||
|
- **候選字**:按 ↓ 選最後一個字或詞,選過的會固定下來
|
||||||
|
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`
|
||||||
|
- **完整詞庫**:約 17 萬筆,來自小麥注音 McBopomofo(MIT 授權)
|
||||||
|
|
||||||
|
## 下載與安裝
|
||||||
|
|
||||||
|
到 Gitea 的 [Releases](https://git.ays.ddnsfree.com/ays953/AutoZhuyin/releases) 頁面下載最新版本。
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
需求:macOS 13 以上,Apple Silicon 和 Intel 都可以(universal binary)。
|
||||||
|
|
||||||
|
1. 雙擊 `AutoZhuyin-<版本>.pkg` 安裝。安裝檔沒有簽章,如果被擋下來,按右鍵 →「打開」,或到「系統設定 → 隱私權與安全性」按「仍要打開」。
|
||||||
|
2. 系統設定 → 鍵盤 → 文字輸入 → 編輯… → + → 繁體中文 → **自動注音**。
|
||||||
|
3. 清單裡沒出現的話,登出再登入一次。
|
||||||
|
|
||||||
|
PKG 會裝到 `/Library/Input Methods/`,系統上所有使用者都能用。
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
需求:Windows 10 / 11,x64、x86、ARM64 都可以。
|
||||||
|
|
||||||
|
1. 先安裝輸入法框架 [PIME](https://github.com/EasyIME/PIME/releases)(免費、開源)。
|
||||||
|
2. 解壓縮 `AutoZhuyin-Windows-<版本>.zip`,在 `install.bat` 上按右鍵 →「以系統管理員身分執行」。
|
||||||
|
3. 設定 → 時間與語言 → 語言與地區 → 中文(台灣) → 語言選項 → 新增鍵盤 → **自動注音 AutoZhuyin**。
|
||||||
|
4. 用 Win + 空白鍵 切換過去。清單裡沒出現的話,重新開機一次。
|
||||||
|
|
||||||
|
SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安裝:以系統管理員身分執行 `uninstall.bat`。
|
||||||
|
|
||||||
|
## 按鍵
|
||||||
|
|
||||||
|
| 按鍵 | 作用 |
|
||||||
|
|---|---|
|
||||||
|
| 空白 | 注音:一聲;音節已完成時送出中文。英文:送出單字加空白 |
|
||||||
|
| Enter | 送出目前組字 |
|
||||||
|
| **Tab** | 判斷錯了?按 Tab 把整段翻轉成另一種語言,輸入法會**記住**這次修正 |
|
||||||
|
| ↓ | 開啟候選字(選最後一個字或詞),數字鍵 1–9 選字,Esc 關閉 |
|
||||||
|
| Esc | 取消組字 |
|
||||||
|
| Shift + 字母 | 強制英文 |
|
||||||
|
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
|
||||||
|
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
|
||||||
|
|
||||||
## 運作方式
|
## 運作方式
|
||||||
|
|
||||||
每按一個鍵,`Classifier` 會同時用兩種解讀幫目前的按鍵序列打分:
|
每按一個鍵,`Classifier` 會同時用兩種解讀幫目前的按鍵序列打分:
|
||||||
@@ -14,84 +63,144 @@ macOS 繁體中文輸入法:**不用按切換鍵**,直接用大千注音鍵
|
|||||||
| 注音分數 | 英文分數 |
|
| 注音分數 | 英文分數 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| 是否能拆成合法注音音節(`SyllableTable`) | 是否為常用英文字(`english-common.txt`) |
|
| 是否能拆成合法注音音節(`SyllableTable`) | 是否為常用英文字(`english-common.txt`) |
|
||||||
| 每個音節有沒有聲調鍵(3 4 6 7 空白) | 是否在系統字典 `/usr/share/dict/words` |
|
| 每個音節有沒有聲調鍵(3 4 6 7 空白) | 是否在英文字典裡 |
|
||||||
| 讀音是否存在於詞庫 | 字母 bigram 模型判斷「像不像英文」 |
|
| 讀音是否存在於詞庫 | 字母 bigram 模型判斷「像不像英文」 |
|
||||||
| 前一段是中文 → 加分(語境) | 前一段是英文 → 加分(語境) |
|
| 前一段是中文 → 加分(語境) | 前一段是英文 → 加分(語境) |
|
||||||
| 使用者過去的修正(學習) | 有大寫字母 → 一定是英文 |
|
| 使用者過去的修正(學習) | 有大寫字母 → 一定是英文 |
|
||||||
|
| 一開始稍微偏向注音 | 純數字 → 數字 |
|
||||||
|
|
||||||
分數高的一方勝出。組字區即時顯示結果(英文以藍字顯示)。
|
分數高的一方勝出。按空白時,注音那邊會把空白當成一聲來評分,英文那邊當作完整單字評分。
|
||||||
|
|
||||||
## 按鍵
|
中英混打時,`Classifier.segment` 會用動態規劃試遍所有切分點,找出總分最高的切法(例如 `ji3hello` → `ji3` 注音+`hello` 英文)。為了避免把正常的單字或句子切碎:
|
||||||
|
- 每多切一刀扣 3 分(`segmentPenalty`)
|
||||||
|
- 後面還接著其他段的注音段,最後一個音節必須打了聲調
|
||||||
|
- 接在英文後面、還沒打聲調的注音段再扣 2 分(`unconfirmedSwitchPenalty`),所以打到 `ji3hel` 時會先顯示「我hel」,不會變成「我heㄠ」
|
||||||
|
|
||||||
| 按鍵 | 作用 |
|
音節解析照大千規則:符號依「聲母 → 介音 → 韻母」填入,聲調鍵結束音節。順序打錯時,如果合併後仍可能是合法音節,就併進同一個音節,否則才切成新音節。
|
||||||
|---|---|
|
|
||||||
| 空白 | 注音:一聲;音節已完成時送出中文。英文:送出單字加空白 |
|
|
||||||
| Enter | 送出目前組字 |
|
|
||||||
| **Tab** | 判斷錯了?按 Tab 在中/英之間翻轉,輸入法會**記住**這次修正 |
|
|
||||||
| ↓ | 開啟候選字(選最後一個字或詞),數字鍵 1–9 選字 |
|
|
||||||
| Esc | 取消組字 |
|
|
||||||
| Shift + 字母 | 強制英文 |
|
|
||||||
| `< > ? ! :` | 中文語境下自動轉成全形 `,。?!:` |
|
|
||||||
|
|
||||||
## 建置與安裝
|
轉換用 Viterbi 演算法,在詞庫中找出整串音節機率最高的詞組組合。
|
||||||
|
|
||||||
需要 Xcode(使用 `/Applications/Xcode.app` 的工具鏈)。
|
## 個人資料
|
||||||
|
|
||||||
|
| | macOS | Windows |
|
||||||
|
|---|---|---|
|
||||||
|
| 學習紀錄(Tab 修正) | `UserDefaults`(`com.autozhuyin.inputmethod.AutoZhuyin`) | `%APPDATA%\PIME\autozhuyin\learned.json` |
|
||||||
|
| 自訂詞 | `~/Library/Application Support/AutoZhuyin/user-phrases.txt` | `%APPDATA%\PIME\autozhuyin\user-phrases.txt` |
|
||||||
|
|
||||||
|
自訂詞的格式和 `lexicon.txt` 一樣,每行一個詞:`ㄗˋ-ㄉㄨㄥˋ-ㄓㄨˋ-ㄧㄣ 自動注音 -4`(讀音、詞、分數;分數越大越優先)。
|
||||||
|
Mac 版可以在輸入法選單選「清除學習紀錄」。
|
||||||
|
|
||||||
|
## 開發
|
||||||
|
|
||||||
|
### macOS
|
||||||
|
|
||||||
|
需要 Xcode。腳本會自動使用 `/Applications/Xcode.app` 的工具鏈。
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
swift test # 執行單元測試(需 DEVELOPER_DIR 指向 Xcode)
|
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
|
||||||
scripts/build.sh # 產生 build/AutoZhuyin.app
|
xcrun swift test # 單元測試
|
||||||
scripts/install.sh # 安裝到 ~/Library/Input Methods 並重啟輸入法
|
scripts/build.sh # 產生 build/AutoZhuyin.app(universal)
|
||||||
|
scripts/install.sh # 裝到 ~/Library/Input Methods、重啟並向系統註冊
|
||||||
|
scripts/package.sh # 產生 build/AutoZhuyin-<版本>.pkg
|
||||||
```
|
```
|
||||||
|
|
||||||
第一次安裝後:系統設定 → 鍵盤 → 文字輸入 → 編輯… → + → 繁體中文 → **自動注音**。
|
改完程式後執行 `scripts/install.sh`,再切換到其他輸入法後切回來,就會用到新版。
|
||||||
如果清單裡沒出現,登出再登入一次。
|
版本號在 `Resources/Info.plist` 的 `CFBundleShortVersionString`。
|
||||||
|
|
||||||
## 詞庫
|
### Windows
|
||||||
|
|
||||||
`Resources/lexicon.txt` 由開源的小麥注音原始資料轉換而來(MIT 授權,約 17 萬筆),
|
Windows 版建立在 [PIME](https://github.com/EasyIME/PIME) 上,核心邏輯用 Python 重寫(`windows/autozhuyin/core/`),行為和測試案例都跟 Mac 版一樣。不需要 Windows 也能開發和測試:
|
||||||
後面再接上手寫的入門詞庫 `Resources/lexicon-seed.txt`。轉換只讀取資料檔,不執行小麥注音的程式碼。
|
|
||||||
要更新到小麥注音最新資料:
|
```bash
|
||||||
|
python3 windows/tools/build_data.py # 產生 windows/autozhuyin/data/
|
||||||
|
(cd windows && python3 -m unittest discover tests) # 核心測試 + 模擬 PIME 的外殼測試
|
||||||
|
scripts/package-windows.sh # 產生 build/AutoZhuyin-Windows-<版本>.zip(會先跑測試)
|
||||||
|
```
|
||||||
|
|
||||||
|
- Windows 沒有內建英文字典,所以 `build_data.py` 會把 macOS 的 `/usr/share/dict/words`(韋氏字典,公有領域)轉成 `english-dict.txt` 一起打包,並預先算好 bigram 表。
|
||||||
|
- `windows/autozhuyin/data/` 是產生出來的,不進版本控制。
|
||||||
|
- 版本號在 `windows/autozhuyin/ime.json`。
|
||||||
|
|
||||||
|
> 修改核心邏輯時,Swift(`Sources/AutoZhuyinCore/`)和 Python(`windows/autozhuyin/core/`)兩邊都要改,並在兩邊的測試加上相同案例。
|
||||||
|
|
||||||
|
### 詞庫
|
||||||
|
|
||||||
|
`Resources/lexicon.txt` 由小麥注音的原始資料轉換而來,後面再接上手寫的入門詞庫 `Resources/lexicon-seed.txt`。轉換程式 `scripts/convert-mcbopomofo.py` 只讀取資料檔,不執行小麥注音的程式碼:分數依語料詞頻計算,一字多音的字依讀音常見程度分配權重。
|
||||||
|
|
||||||
|
更新到小麥注音最新資料:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
scripts/import-mcbopomofo.sh
|
scripts/import-mcbopomofo.sh
|
||||||
scripts/install.sh
|
|
||||||
```
|
```
|
||||||
|
|
||||||
自訂詞:`~/Library/Application Support/AutoZhuyin/user-phrases.txt`,格式同 `lexicon.txt`:
|
### 發布新版本
|
||||||
`ㄗˋ-ㄉㄨㄥˋ-ㄓㄨˋ-ㄧㄣ 自動注音 -4`
|
|
||||||
|
先 commit 所有修改,再執行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/release.sh # 修正版:0.1.0 → 0.1.1
|
||||||
|
scripts/release.sh minor # 新功能:0.1.1 → 0.2.0
|
||||||
|
scripts/release.sh major # 大改版:0.2.0 → 1.0.0
|
||||||
|
scripts/release.sh minor --push # 同時 push commit 和 tag
|
||||||
|
```
|
||||||
|
|
||||||
|
`release.sh` 會自動:
|
||||||
|
1. 遞增版本號(`Resources/Info.plist` 的版本與 build 號、`windows/autozhuyin/ime.json`)
|
||||||
|
2. 跑 Swift 和 Python 全部測試
|
||||||
|
3. 產生 `build/AutoZhuyin-<版本>.pkg` 和 `build/AutoZhuyin-Windows-<版本>.zip`
|
||||||
|
4. commit「Release v<版本>」並打上 `v<版本>` tag
|
||||||
|
|
||||||
|
任何一步失敗都會把版本號改回去。只想改版本號不打包:`scripts/bump-version.sh [patch|minor|major]`。
|
||||||
|
|
||||||
|
最後在 Gitea 用這個 tag 建立 Release,附上 `build/` 裡的 `.pkg` 和 `.zip`。
|
||||||
|
|
||||||
## 專案結構
|
## 專案結構
|
||||||
|
|
||||||
```
|
```
|
||||||
Sources/AutoZhuyinCore/ 純邏輯,可單元測試
|
Sources/AutoZhuyinCore/ 核心邏輯(Swift,可單元測試)
|
||||||
Bopomofo.swift 大千鍵盤對應、音節結構
|
Bopomofo.swift 大千鍵盤對應、音節結構
|
||||||
SyllableTable.swift 合法注音音節表
|
SyllableTable.swift 合法注音音節表
|
||||||
ZhuyinParser.swift 按鍵 → 音節
|
ZhuyinParser.swift 按鍵 → 音節(含順序錯誤修正)
|
||||||
Lexicon.swift 詞庫
|
Lexicon.swift 詞庫
|
||||||
Converter.swift 音節 → 中文(Viterbi)
|
Converter.swift 音節 → 中文(Viterbi)
|
||||||
EnglishModel.swift 英文判斷
|
EnglishModel.swift 英文判斷
|
||||||
Classifier.swift 中英判斷 + 學習
|
Classifier.swift 中英判斷 + 學習
|
||||||
CompositionEngine.swift 輸入法狀態機
|
CompositionEngine.swift 輸入法狀態機
|
||||||
Sources/AutoZhuyin/ InputMethodKit 外殼
|
Sources/AutoZhuyin/ macOS InputMethodKit 外殼
|
||||||
Tests/ 單元測試
|
Tests/ Swift 單元測試
|
||||||
|
Resources/ Info.plist、詞庫、英文常用字
|
||||||
|
scripts/ 建置、安裝、打包、詞庫轉換
|
||||||
|
pkg/postinstall PKG 安裝後腳本
|
||||||
|
windows/
|
||||||
|
autozhuyin/ PIME 輸入法(整個資料夾會被複製到 PIME)
|
||||||
|
core/ 核心邏輯(Python 版)
|
||||||
|
autozhuyin_ime.py PIME 外殼
|
||||||
|
ime.json PIME 輸入法設定
|
||||||
|
installer/ install.bat、uninstall.bat、給使用者的 README.txt
|
||||||
|
tests/ Python 測試
|
||||||
|
tools/ 產生資料檔、圖示
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 已知限制
|
||||||
|
|
||||||
|
- Windows 版還沒在實機上測試過。
|
||||||
|
- 聲調鍵打在注音符號中間(例如 `uv3l`)無法自動修正。
|
||||||
|
- 很短的字串(如 `up` 是英文還是 ㄧㄣ)本質上就有歧義,可能需要按 Tab 修正;修正後會記住。
|
||||||
|
- 安裝檔沒有簽章(macOS 沒有 Developer ID,Windows 沒有程式碼簽章憑證)。
|
||||||
|
|
||||||
## 下一步(Roadmap)
|
## 下一步(Roadmap)
|
||||||
|
|
||||||
- [ ] 匯入完整詞庫,並用真實打字資料調整判斷權重
|
- [x] 匯入完整詞庫
|
||||||
|
- [x] 注音順序錯誤修正
|
||||||
|
- [x] Windows 版
|
||||||
|
- [ ] 用真實打字資料調整判斷權重
|
||||||
- [ ] 已送出的英文單字若判斷錯誤,可以「事後翻轉」
|
- [ ] 已送出的英文單字若判斷錯誤,可以「事後翻轉」
|
||||||
- [ ] 用 Core ML 小模型取代手調分數,參考整句語境
|
- [ ] 用 Core ML 小模型取代手調分數,參考整句語境
|
||||||
- [ ] 使用者詞頻學習(選過的候選字優先)
|
- [ ] 使用者詞頻學習(選過的候選字優先)
|
||||||
- [ ] 偏好設定視窗
|
- [ ] 偏好設定視窗
|
||||||
|
|
||||||
## Windows 版
|
## 致謝與授權
|
||||||
|
|
||||||
`windows/` 是給 Windows 用的版本,建立在開源輸入法框架 [PIME](https://github.com/EasyIME/PIME) 上。
|
- 詞庫資料:[小麥注音 McBopomofo](https://github.com/openvanilla/McBopomofo)(MIT License)
|
||||||
核心邏輯用 Python 重寫(`windows/autozhuyin/core/`),行為和測試案例都跟 Mac 版一樣;詞庫共用 `Resources/`。
|
- Windows 輸入法框架:[PIME](https://github.com/EasyIME/PIME)(LGPL 2.1,使用者自行安裝,本專案不包含其程式碼)
|
||||||
|
- 英文字典:Webster's Second International Dictionary(公有領域,macOS 的 `/usr/share/dict/words`)
|
||||||
```bash
|
|
||||||
scripts/package-windows.sh # 產生 build/AutoZhuyin-Windows-<版本>.zip(會先跑測試)
|
|
||||||
```
|
|
||||||
|
|
||||||
zip 裡附有 `install.bat`、`uninstall.bat` 和給測試者看的 `README.txt`。使用者需要先安裝 PIME。
|
|
||||||
修改核心邏輯時,Swift(`Sources/AutoZhuyinCore/`)和 Python(`windows/autozhuyin/core/`)兩邊都要改。
|
|
||||||
|
|||||||
+14
-14
@@ -4,42 +4,42 @@
|
|||||||
<dict>
|
<dict>
|
||||||
<key>CFBundleDevelopmentRegion</key>
|
<key>CFBundleDevelopmentRegion</key>
|
||||||
<string>zh_TW</string>
|
<string>zh_TW</string>
|
||||||
|
<key>CFBundleDisplayName</key>
|
||||||
|
<string>自動注音</string>
|
||||||
<key>CFBundleExecutable</key>
|
<key>CFBundleExecutable</key>
|
||||||
<string>AutoZhuyin</string>
|
<string>AutoZhuyin</string>
|
||||||
<key>CFBundleIdentifier</key>
|
<key>CFBundleIdentifier</key>
|
||||||
<string>com.autozhuyin.inputmethod.AutoZhuyin</string>
|
<string>com.autozhuyin.inputmethod.AutoZhuyin</string>
|
||||||
<key>CFBundleName</key>
|
<key>CFBundleName</key>
|
||||||
<string>AutoZhuyin</string>
|
<string>AutoZhuyin</string>
|
||||||
<key>CFBundleDisplayName</key>
|
|
||||||
<string>自動注音</string>
|
|
||||||
<key>CFBundlePackageType</key>
|
<key>CFBundlePackageType</key>
|
||||||
<string>APPL</string>
|
<string>APPL</string>
|
||||||
<key>CFBundleShortVersionString</key>
|
<key>CFBundleShortVersionString</key>
|
||||||
<string>0.1.0</string>
|
<string>0.2.0</string>
|
||||||
<key>CFBundleVersion</key>
|
<key>CFBundleVersion</key>
|
||||||
<string>1</string>
|
<string>2</string>
|
||||||
<key>LSMinimumSystemVersion</key>
|
|
||||||
<string>13.0</string>
|
|
||||||
<key>LSBackgroundOnly</key>
|
|
||||||
<true/>
|
|
||||||
<key>LSUIElement</key>
|
|
||||||
<true/>
|
|
||||||
<key>NSPrincipalClass</key>
|
|
||||||
<string>NSApplication</string>
|
|
||||||
<key>InputMethodConnectionName</key>
|
<key>InputMethodConnectionName</key>
|
||||||
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
||||||
<key>InputMethodServerControllerClass</key>
|
<key>InputMethodServerControllerClass</key>
|
||||||
<string>AutoZhuyinInputController</string>
|
<string>AutoZhuyinInputController</string>
|
||||||
|
<key>LSBackgroundOnly</key>
|
||||||
|
<true/>
|
||||||
|
<key>LSMinimumSystemVersion</key>
|
||||||
|
<string>13.0</string>
|
||||||
|
<key>LSUIElement</key>
|
||||||
|
<true/>
|
||||||
|
<key>NSPrincipalClass</key>
|
||||||
|
<string>NSApplication</string>
|
||||||
<key>TISInputSourceID</key>
|
<key>TISInputSourceID</key>
|
||||||
<string>com.autozhuyin.inputmethod.AutoZhuyin</string>
|
<string>com.autozhuyin.inputmethod.AutoZhuyin</string>
|
||||||
<key>TISIntendedLanguage</key>
|
<key>TISIntendedLanguage</key>
|
||||||
<string>zh-Hant</string>
|
<string>zh-Hant</string>
|
||||||
<key>tsInputMethodIconFileKey</key>
|
|
||||||
<string>icon.tiff</string>
|
|
||||||
<key>tsInputMethodCharacterRepertoireKey</key>
|
<key>tsInputMethodCharacterRepertoireKey</key>
|
||||||
<array>
|
<array>
|
||||||
<string>Hant</string>
|
<string>Hant</string>
|
||||||
<string>Latn</string>
|
<string>Latn</string>
|
||||||
</array>
|
</array>
|
||||||
|
<key>tsInputMethodIconFileKey</key>
|
||||||
|
<string>icon.tiff</string>
|
||||||
</dict>
|
</dict>
|
||||||
</plist>
|
</plist>
|
||||||
|
|||||||
@@ -90,13 +90,15 @@ final class InputController: IMKInputController {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private func updateComposition(_ client: IMKTextInput) {
|
private func updateComposition(_ client: IMKTextInput) {
|
||||||
let text = engine.markedText
|
// Each segment is underlined; literal (English) segments are blue as a hint.
|
||||||
let attrs: [NSAttributedString.Key: Any] = [
|
let marked = NSMutableAttributedString()
|
||||||
|
for piece in engine.markedPieces {
|
||||||
|
marked.append(NSAttributedString(string: piece.text, attributes: [
|
||||||
.underlineStyle: NSUnderlineStyle.single.rawValue,
|
.underlineStyle: NSUnderlineStyle.single.rawValue,
|
||||||
// Literal (English) composition is shown in a different colour as a hint.
|
.foregroundColor: piece.mode == .literal ? NSColor.systemBlue : NSColor.textColor,
|
||||||
.foregroundColor: engine.mode == .literal ? NSColor.systemBlue : NSColor.textColor,
|
]))
|
||||||
]
|
}
|
||||||
let marked = NSAttributedString(string: text, attributes: text.isEmpty ? [:] : attrs)
|
let text = marked.string
|
||||||
client.setMarkedText(marked,
|
client.setMarkedText(marked,
|
||||||
selectionRange: NSRange(location: (text as NSString).length, length: 0),
|
selectionRange: NSRange(location: (text as NSString).length, length: 0),
|
||||||
replacementRange: NSRange(location: NSNotFound, length: 0))
|
replacementRange: NSRange(location: NSNotFound, length: 0))
|
||||||
|
|||||||
@@ -7,6 +7,17 @@ public enum InputMode: String, Codable {
|
|||||||
case literal
|
case literal
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A run of keys (indices into the buffer) interpreted in one mode.
|
||||||
|
public struct Segment: Equatable {
|
||||||
|
public var range: Range<Int>
|
||||||
|
public var mode: InputMode
|
||||||
|
|
||||||
|
public init(range: Range<Int>, mode: InputMode) {
|
||||||
|
self.range = range
|
||||||
|
self.mode = mode
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public struct Classification {
|
public struct Classification {
|
||||||
public var mode: InputMode
|
public var mode: InputMode
|
||||||
public var zhuyinScore: Double
|
public var zhuyinScore: Double
|
||||||
@@ -72,6 +83,83 @@ public final class Classifier {
|
|||||||
return Classification(mode: zh >= lit ? .zhuyin : .literal, zhuyinScore: zh, literalScore: lit)
|
return Classification(mode: zh >= lit ? .zhuyin : .literal, zhuyinScore: zh, literalScore: lit)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// MARK: - Segmentation
|
||||||
|
|
||||||
|
/// Cost of each switch between Zhuyin and literal inside one buffer, so that a normal
|
||||||
|
/// word or sentence is not chopped up.
|
||||||
|
public var segmentPenalty = 3.0
|
||||||
|
|
||||||
|
/// Splits a buffer into alternating Zhuyin / literal segments, e.g. "ji3hello" →
|
||||||
|
/// [ji3: zhuyin, hello: literal]. Dynamic programming over every split point.
|
||||||
|
/// A Zhuyin segment followed by another segment must end with a tone key.
|
||||||
|
public func segment(_ keys: [Character], finishing: Bool = false, previous: InputMode? = nil) -> [Segment] {
|
||||||
|
let n = keys.count
|
||||||
|
guard n > 0 else { return [] }
|
||||||
|
let modes: [InputMode] = [.zhuyin, .literal]
|
||||||
|
// best[i][m]: best score for keys[0..<i] whose last segment has mode m.
|
||||||
|
var best = Array(repeating: [Double](repeating: -.infinity, count: 2), count: n + 1)
|
||||||
|
var back = Array(repeating: [Int](repeating: -1, count: 2), count: n + 1) // segment start j
|
||||||
|
|
||||||
|
for i in 1...n {
|
||||||
|
for j in 0..<i {
|
||||||
|
let scores = segmentScores(Array(keys[j..<i]), isLast: i == n, afterOther: j > 0,
|
||||||
|
finishing: finishing, previous: j == 0 ? previous : nil,
|
||||||
|
whole: j == 0 && i == n)
|
||||||
|
for m in 0..<2 where scores[m] > -.infinity {
|
||||||
|
// Adjacent segments must alternate modes; same-mode neighbours are one segment.
|
||||||
|
let total = j == 0 ? scores[m] : best[j][1 - m] + scores[m] - segmentPenalty
|
||||||
|
if total > best[i][m] {
|
||||||
|
best[i][m] = total
|
||||||
|
back[i][m] = j
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
guard best[n][0] > -.infinity || best[n][1] > -.infinity else {
|
||||||
|
return [Segment(range: 0..<n, mode: classify(keys, finishing: finishing, previous: previous).mode)]
|
||||||
|
}
|
||||||
|
var m = best[n][0] >= best[n][1] ? 0 : 1
|
||||||
|
var i = n
|
||||||
|
var out: [Segment] = []
|
||||||
|
while i > 0 {
|
||||||
|
let j = back[i][m]
|
||||||
|
out.append(Segment(range: j..<i, mode: modes[m]))
|
||||||
|
i = j
|
||||||
|
m = 1 - m
|
||||||
|
}
|
||||||
|
return out.reversed()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Extra cost for a trailing Zhuyin segment after English that has no tone key yet
|
||||||
|
/// ("ji3hel" should stay 我hel, not become 我heㄠ, until a tone confirms the switch).
|
||||||
|
public var unconfirmedSwitchPenalty = 2.0
|
||||||
|
|
||||||
|
private func segmentScores(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
|
||||||
|
previous: InputMode?, whole: Bool) -> [Double] {
|
||||||
|
var zh: Double
|
||||||
|
let parse = ZhuyinParser.parse(seg)
|
||||||
|
if !isLast && parse.pending != nil {
|
||||||
|
zh = -.infinity
|
||||||
|
} else {
|
||||||
|
zh = zhuyinScore(finishing && isLast ? seg + [" "] : seg) + zhuyinPrior
|
||||||
|
if isLast && afterOther && !finishing && !parse.syllables.contains(where: \.toned) {
|
||||||
|
zh -= unconfirmedSwitchPenalty
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var lit = literalScore(seg, complete: !isLast || finishing)
|
||||||
|
switch previous {
|
||||||
|
case .zhuyin?: zh += contextWeight
|
||||||
|
case .literal?: lit += contextWeight
|
||||||
|
case nil: break
|
||||||
|
}
|
||||||
|
if whole {
|
||||||
|
let b = learned.bias(for: String(seg))
|
||||||
|
if b > 0 { zh += b } else { lit -= b }
|
||||||
|
}
|
||||||
|
return [zh, lit]
|
||||||
|
}
|
||||||
|
|
||||||
public func zhuyinScore(_ keys: [Character]) -> Double {
|
public func zhuyinScore(_ keys: [Character]) -> Double {
|
||||||
let parse = ZhuyinParser.parse(keys)
|
let parse = ZhuyinParser.parse(keys)
|
||||||
if parse.foreignKeys > 0 || parse.syllables.isEmpty { return -.infinity }
|
if parse.foreignKeys > 0 || parse.syllables.isEmpty { return -.infinity }
|
||||||
|
|||||||
@@ -16,22 +16,29 @@ public struct KeyResult: Equatable {
|
|||||||
public static let consumed = KeyResult(handled: true, commit: nil)
|
public static let consumed = KeyResult(handled: true, commit: nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A piece of the composition as shown to the user.
|
||||||
|
public struct MarkedPiece: Equatable {
|
||||||
|
public var text: String
|
||||||
|
public var mode: InputMode
|
||||||
|
}
|
||||||
|
|
||||||
/// The whole input-method state machine, independent of InputMethodKit.
|
/// The whole input-method state machine, independent of InputMethodKit.
|
||||||
///
|
///
|
||||||
/// Keystrokes accumulate in a buffer. On every key the classifier decides whether the
|
/// Keystrokes accumulate in a buffer. On every key the classifier splits the buffer
|
||||||
/// buffer is Zhuyin or literal text; the composition (marked text) shows the result
|
/// into Zhuyin and literal segments (e.g. "ji3hello" → 我 + hello); the composition
|
||||||
/// live. Space / Enter / punctuation commit it. Tab flips the decision and the
|
/// (marked text) shows the result live. Space / Enter / punctuation commit it. Tab
|
||||||
/// correction is learned.
|
/// forces the whole buffer into the other mode and the correction is learned.
|
||||||
public final class CompositionEngine {
|
public final class CompositionEngine {
|
||||||
public let classifier: Classifier
|
public let classifier: Classifier
|
||||||
public let converter: Converter
|
public let converter: Converter
|
||||||
|
|
||||||
public private(set) var keys: [Character] = []
|
public private(set) var keys: [Character] = []
|
||||||
/// Set by Tab: overrides the classifier for the current buffer.
|
/// Set by Tab: overrides the classifier for the whole buffer.
|
||||||
public private(set) var forcedMode: InputMode?
|
public private(set) var forcedMode: InputMode?
|
||||||
/// User-picked phrases, by syllable range.
|
/// User-picked phrases: Zhuyin segment start (key index) → syllable range → text.
|
||||||
public private(set) var fixed: [Range<Int>: String] = [:]
|
private var fixed: [Int: [Range<Int>: String]] = [:]
|
||||||
public private(set) var candidates: [Candidate] = []
|
public private(set) var candidates: [Candidate] = []
|
||||||
|
private var candidateSegmentStart = 0
|
||||||
public var candidateIndex = 0
|
public var candidateIndex = 0
|
||||||
public var isShowingCandidates: Bool { !candidates.isEmpty }
|
public var isShowingCandidates: Bool { !candidates.isEmpty }
|
||||||
|
|
||||||
@@ -52,33 +59,41 @@ public final class CompositionEngine {
|
|||||||
now().timeIntervalSince(previousCommitTime) < contextWindow ? previousMode : nil
|
now().timeIntervalSince(previousCommitTime) < contextWindow ? previousMode : nil
|
||||||
}
|
}
|
||||||
|
|
||||||
public var mode: InputMode {
|
private func segments(of keys: [Character], finishing: Bool = false) -> [Segment] {
|
||||||
forcedMode ?? classifier.classify(keys, previous: context).mode
|
guard !keys.isEmpty else { return [] }
|
||||||
|
if let forced = forcedMode { return [Segment(range: 0..<keys.count, mode: forced)] }
|
||||||
|
return classifier.segment(keys, finishing: finishing, previous: context)
|
||||||
}
|
}
|
||||||
|
|
||||||
public var parse: ZhuyinParse { ZhuyinParser.parse(keys) }
|
/// Current split of the buffer into Zhuyin / literal runs.
|
||||||
|
public var segments: [Segment] { segments(of: keys) }
|
||||||
|
|
||||||
|
/// Mode of the last segment (what the user is typing right now).
|
||||||
|
public var mode: InputMode { segments.last?.mode ?? .zhuyin }
|
||||||
|
|
||||||
/// What to show underlined in the app while composing.
|
/// What to show underlined in the app while composing.
|
||||||
public var markedText: String {
|
public var markedText: String { markedPieces.map(\.text).joined() }
|
||||||
guard !keys.isEmpty else { return "" }
|
|
||||||
switch mode {
|
public var markedPieces: [MarkedPiece] {
|
||||||
case .literal:
|
let segs = segments
|
||||||
return String(keys)
|
return segs.enumerated().map { i, seg in
|
||||||
case .zhuyin:
|
MarkedPiece(text: render(seg, final: i < segs.count - 1), mode: seg.mode)
|
||||||
let p = parse
|
|
||||||
let done = converter.convert(p.completed.map(\.syllable), fixed: validFixed(p)).map(\.text).joined()
|
|
||||||
return done + (p.pending?.syllable.reading ?? "")
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private func validFixed(_ p: ZhuyinParse) -> [Range<Int>: String] {
|
/// Text for one segment. A non-final Zhuyin segment shows its unfinished syllable as Bopomofo.
|
||||||
fixed.filter { $0.key.upperBound <= p.completed.count }
|
private func render(_ seg: Segment, final: Bool) -> String {
|
||||||
|
let segKeys = Array(keys[seg.range])
|
||||||
|
guard seg.mode == .zhuyin else { return String(segKeys) }
|
||||||
|
let p = ZhuyinParser.parse(segKeys)
|
||||||
|
let fixedHere = (fixed[seg.range.lowerBound] ?? [:])
|
||||||
|
if final {
|
||||||
|
let all = p.syllables.map(\.syllable)
|
||||||
|
return converter.convert(all, fixed: fixedHere.filter { $0.key.upperBound <= all.count }).map(\.text).joined()
|
||||||
}
|
}
|
||||||
|
let done = p.completed.map(\.syllable)
|
||||||
private func zhuyinCommitText() -> String {
|
let text = converter.convert(done, fixed: fixedHere.filter { $0.key.upperBound <= done.count }).map(\.text).joined()
|
||||||
let p = parse
|
return text + (p.pending?.syllable.reading ?? "")
|
||||||
// An unfinished trailing syllable is converted as untoned.
|
|
||||||
return converter.convert(p.syllables.map(\.syllable), fixed: validFixed(p)).map(\.text).joined()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// MARK: - Key handling
|
// MARK: - Key handling
|
||||||
@@ -92,7 +107,7 @@ public final class CompositionEngine {
|
|||||||
case .backspace:
|
case .backspace:
|
||||||
guard !keys.isEmpty else { return .passThrough }
|
guard !keys.isEmpty else { return .passThrough }
|
||||||
keys.removeLast()
|
keys.removeLast()
|
||||||
fixed = validFixed(parse)
|
fixed = fixed.filter { $0.key < keys.count }
|
||||||
if keys.isEmpty { forcedMode = nil; fixed = [:] }
|
if keys.isEmpty { forcedMode = nil; fixed = [:] }
|
||||||
return .consumed
|
return .consumed
|
||||||
case .enter:
|
case .enter:
|
||||||
@@ -118,15 +133,15 @@ public final class CompositionEngine {
|
|||||||
private func handleCharacter(_ c: Character) -> KeyResult {
|
private func handleCharacter(_ c: Character) -> KeyResult {
|
||||||
if c == " " {
|
if c == " " {
|
||||||
guard !keys.isEmpty else { return .passThrough }
|
guard !keys.isEmpty else { return .passThrough }
|
||||||
let decided = forcedMode ?? classifier.classify(keys, finishing: true, previous: context).mode
|
let segs = segments(of: keys, finishing: true)
|
||||||
if decided == .literal {
|
if segs.last?.mode == .literal {
|
||||||
return KeyResult(handled: true, commit: commitCurrent(as: .literal) + " ")
|
return KeyResult(handled: true, commit: commitCurrent(segs) + " ")
|
||||||
}
|
}
|
||||||
if parse.pending != nil {
|
if parse(ofLast: segs)?.pending != nil {
|
||||||
keys.append(" ") // first tone
|
keys.append(" ") // first tone
|
||||||
return .consumed
|
return .consumed
|
||||||
}
|
}
|
||||||
return KeyResult(handled: true, commit: commitCurrent(as: .zhuyin))
|
return KeyResult(handled: true, commit: commitCurrent(segs))
|
||||||
}
|
}
|
||||||
|
|
||||||
let composable = DachenLayout.isZhuyinKey(c) || (c.isASCII && c.isLetter) || (c == "'" && !keys.isEmpty)
|
let composable = DachenLayout.isZhuyinKey(c) || (c.isASCII && c.isLetter) || (c == "'" && !keys.isEmpty)
|
||||||
@@ -134,9 +149,9 @@ public final class CompositionEngine {
|
|||||||
// ", . / ; -" are Zhuyin symbols (ㄝ ㄡ ㄥ ㄤ ㄦ) but act as punctuation after an
|
// ", . / ; -" are Zhuyin symbols (ㄝ ㄡ ㄥ ㄤ ㄦ) but act as punctuation after an
|
||||||
// English word. Decide with the key included: "2;" is ㄉㄤ (檔…), "hello," is English.
|
// English word. Decide with the key included: "2;" is ㄉㄤ (檔…), "hello," is English.
|
||||||
if !keys.isEmpty, DachenLayout.symbols[c] != nil, !c.isLetter, !c.isNumber, forcedMode != .zhuyin {
|
if !keys.isEmpty, DachenLayout.symbols[c] != nil, !c.isLetter, !c.isNumber, forcedMode != .zhuyin {
|
||||||
let withKey = forcedMode ?? classifier.classify(keys + [c], previous: context).mode
|
let withKey = segments(of: keys + [c])
|
||||||
if withKey == .literal && !keys.allSatisfy(\.isNumber) {
|
if withKey.last?.mode == .literal && !keys.allSatisfy(\.isNumber) {
|
||||||
return KeyResult(handled: true, commit: commitCurrent(as: .literal) + String(c))
|
return KeyResult(handled: true, commit: commitCurrent() + String(c))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
keys.append(c)
|
keys.append(c)
|
||||||
@@ -151,25 +166,34 @@ public final class CompositionEngine {
|
|||||||
return KeyResult(handled: true, commit: committed + punct)
|
return KeyResult(handled: true, commit: committed + punct)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private func parse(ofLast segs: [Segment]) -> ZhuyinParse? {
|
||||||
|
guard let last = segs.last, last.mode == .zhuyin else { return nil }
|
||||||
|
return ZhuyinParser.parse(Array(keys[last.range]))
|
||||||
|
}
|
||||||
|
|
||||||
static let fullwidth: [Character: String] = [
|
static let fullwidth: [Character: String] = [
|
||||||
"<": ",", ">": "。", "?": "?", "!": "!", ":": ":", "\"": "「", "(": "(", ")": ")",
|
"<": ",", ">": "。", "?": "?", "!": "!", ":": ":", "\"": "「", "(": "(", ")": ")",
|
||||||
"{": "『", "}": "』", "[": "「", "]": "」", "~": "~", "_": "——",
|
"{": "『", "}": "』", "[": "「", "]": "」", "~": "~", "_": "——",
|
||||||
]
|
]
|
||||||
|
|
||||||
// MARK: - Candidates
|
// MARK: - Candidates (for the last Zhuyin segment)
|
||||||
|
|
||||||
private func openCandidates() {
|
private func openCandidates() {
|
||||||
let p = parse
|
let segs = segments
|
||||||
guard mode == .zhuyin, !p.completed.isEmpty else { return }
|
guard let last = segs.last, last.mode == .zhuyin else { return }
|
||||||
candidates = converter.candidates(p.completed.map(\.syllable), endingAt: p.completed.count)
|
let done = ZhuyinParser.parse(Array(keys[last.range])).completed.map(\.syllable)
|
||||||
|
guard !done.isEmpty else { return }
|
||||||
|
candidates = converter.candidates(done, endingAt: done.count)
|
||||||
|
candidateSegmentStart = last.range.lowerBound
|
||||||
candidateIndex = 0
|
candidateIndex = 0
|
||||||
}
|
}
|
||||||
|
|
||||||
public func selectCandidate(_ index: Int) {
|
public func selectCandidate(_ index: Int) {
|
||||||
guard candidates.indices.contains(index) else { return }
|
guard candidates.indices.contains(index) else { return }
|
||||||
let c = candidates[index]
|
let c = candidates[index]
|
||||||
fixed = fixed.filter { !$0.key.overlaps(c.range) }
|
var here = (fixed[candidateSegmentStart] ?? [:]).filter { !$0.key.overlaps(c.range) }
|
||||||
fixed[c.range] = c.text
|
here[c.range] = c.text
|
||||||
|
fixed[candidateSegmentStart] = here
|
||||||
candidates = []
|
candidates = []
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -197,13 +221,13 @@ public final class CompositionEngine {
|
|||||||
// MARK: - Commit
|
// MARK: - Commit
|
||||||
|
|
||||||
@discardableResult
|
@discardableResult
|
||||||
private func commitCurrent(as decided: InputMode? = nil) -> String {
|
private func commitCurrent(_ given: [Segment]? = nil) -> String {
|
||||||
let m = decided ?? mode
|
let segs = given ?? segments
|
||||||
let text = m == .literal ? String(keys) : zhuyinCommitText()
|
let text = segs.map { render($0, final: true) }.joined()
|
||||||
if let forced = forcedMode {
|
if let forced = forcedMode {
|
||||||
classifier.learned.record(String(keys), as: forced)
|
classifier.learned.record(String(keys), as: forced)
|
||||||
}
|
}
|
||||||
previousMode = m
|
previousMode = segs.last?.mode
|
||||||
previousCommitTime = now()
|
previousCommitTime = now()
|
||||||
reset()
|
reset()
|
||||||
return text
|
return text
|
||||||
|
|||||||
@@ -219,3 +219,37 @@ final class DigitInitialTests: XCTestCase {
|
|||||||
XCTAssertEqual(run("hello, ").out, "hello, ")
|
XCTAssertEqual(run("hello, ").out, "hello, ")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
final class MixedSegmentTests: XCTestCase {
|
||||||
|
func run(_ s: String) -> (out: String, marked: String) {
|
||||||
|
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish))
|
||||||
|
var out = ""
|
||||||
|
for c in s {
|
||||||
|
let r = e.handle(c == "\n" ? .enter : .character(c))
|
||||||
|
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
|
||||||
|
}
|
||||||
|
return (out, e.markedText)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testChineseThenEnglish() {
|
||||||
|
XCTAssertEqual(run("ji3hello ").out, "我hello ")
|
||||||
|
XCTAssertEqual(run("su3cl3github\n").out, "你好github")
|
||||||
|
XCTAssertEqual(run("ji3hel").marked, "我hel")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testEnglishThenChinese() {
|
||||||
|
XCTAssertEqual(run("hellosu3cl3 ").out, "hello你好")
|
||||||
|
XCTAssertEqual(run("oksu3cl3\n").out, "ok你好")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testThreeSegments() {
|
||||||
|
XCTAssertEqual(run("ji3emailsu3cl3\n").out, "我email你好")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testSingleLanguageUnchanged() {
|
||||||
|
XCTAssertEqual(run("rup wu0 wu0 fu4cp3cl3\n").out, "今天天氣很好")
|
||||||
|
XCTAssertEqual(run("helloworld\n").out, "helloworld")
|
||||||
|
XCTAssertEqual(run("meeting ").out, "meeting ")
|
||||||
|
XCTAssertEqual(run("183\n").out.count, 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
Executable
+32
@@ -0,0 +1,32 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Increments the version in every place it lives:
|
||||||
|
# Resources/Info.plist CFBundleShortVersionString (x.y.z) and CFBundleVersion (build number)
|
||||||
|
# windows/autozhuyin/ime.json "version"
|
||||||
|
#
|
||||||
|
# Usage: scripts/bump-version.sh [patch|minor|major] (default: patch)
|
||||||
|
# Prints the new version.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
LEVEL="${1:-patch}"
|
||||||
|
PLIST=Resources/Info.plist
|
||||||
|
IME=windows/autozhuyin/ime.json
|
||||||
|
|
||||||
|
CURRENT="$(plutil -extract CFBundleShortVersionString raw "$PLIST")"
|
||||||
|
IFS=. read -r MAJOR MINOR PATCH <<< "$CURRENT"
|
||||||
|
case "$LEVEL" in
|
||||||
|
major) MAJOR=$((MAJOR + 1)); MINOR=0; PATCH=0 ;;
|
||||||
|
minor) MINOR=$((MINOR + 1)); PATCH=0 ;;
|
||||||
|
patch) PATCH=$((PATCH + 1)) ;;
|
||||||
|
*) echo "usage: $0 [patch|minor|major]" >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
NEW="$MAJOR.$MINOR.$PATCH"
|
||||||
|
BUILD=$(( $(plutil -extract CFBundleVersion raw "$PLIST") + 1 ))
|
||||||
|
|
||||||
|
plutil -replace CFBundleShortVersionString -string "$NEW" "$PLIST"
|
||||||
|
plutil -replace CFBundleVersion -string "$BUILD" "$PLIST"
|
||||||
|
# Edit only the version line so the rest of ime.json keeps its formatting.
|
||||||
|
sed -i '' -E "s/(\"version\"[[:space:]]*:[[:space:]]*\")[^\"]*\"/\1$NEW\"/" "$IME"
|
||||||
|
|
||||||
|
grep -q "\"version\": \"$NEW\"" "$IME" || { echo "failed to update $IME" >&2; exit 1; }
|
||||||
|
echo "$NEW"
|
||||||
Executable
+54
@@ -0,0 +1,54 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# One-step release: bump the version, run all tests, build both installers, then
|
||||||
|
# commit the version change and tag it.
|
||||||
|
#
|
||||||
|
# Usage: scripts/release.sh [patch|minor|major] [--push] (default: patch)
|
||||||
|
# --push also pushes the commit and tag to origin
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
LEVEL=patch
|
||||||
|
PUSH=0
|
||||||
|
for arg in "$@"; do
|
||||||
|
case "$arg" in
|
||||||
|
patch|minor|major) LEVEL="$arg" ;;
|
||||||
|
--push) PUSH=1 ;;
|
||||||
|
*) echo "usage: $0 [patch|minor|major] [--push]" >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ -n "$(git status --porcelain)" ]; then
|
||||||
|
echo "Working tree has uncommitted changes; commit them first." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
export DEVELOPER_DIR="${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}"
|
||||||
|
|
||||||
|
VERSION="$(scripts/bump-version.sh "$LEVEL")"
|
||||||
|
echo "==> Version $VERSION"
|
||||||
|
|
||||||
|
restore() {
|
||||||
|
echo "Release failed; reverting the version bump." >&2
|
||||||
|
git checkout -- Resources/Info.plist windows/autozhuyin/ime.json
|
||||||
|
}
|
||||||
|
trap restore ERR
|
||||||
|
|
||||||
|
echo "==> Swift tests"
|
||||||
|
xcrun swift test -q 2>&1 | tail -3
|
||||||
|
echo "==> Packaging (runs the Python tests too)"
|
||||||
|
scripts/package.sh >/dev/null
|
||||||
|
scripts/package-windows.sh >/dev/null
|
||||||
|
|
||||||
|
trap - ERR
|
||||||
|
git add Resources/Info.plist windows/autozhuyin/ime.json
|
||||||
|
git commit -q -m "Release v$VERSION"
|
||||||
|
git tag -a "v$VERSION" -m "AutoZhuyin v$VERSION"
|
||||||
|
echo "==> Tagged v$VERSION"
|
||||||
|
ls -lh "build/AutoZhuyin-$VERSION.pkg" "build/AutoZhuyin-Windows-$VERSION.zip"
|
||||||
|
shasum -a 256 "build/AutoZhuyin-$VERSION.pkg" "build/AutoZhuyin-Windows-$VERSION.zip"
|
||||||
|
|
||||||
|
if [ "$PUSH" = 1 ]; then
|
||||||
|
git push origin HEAD "v$VERSION"
|
||||||
|
else
|
||||||
|
echo "Push with: git push origin HEAD v$VERSION"
|
||||||
|
fi
|
||||||
@@ -55,6 +55,69 @@ class Classifier:
|
|||||||
def mode(self, keys, finishing=False, previous=None):
|
def mode(self, keys, finishing=False, previous=None):
|
||||||
return self.classify(keys, finishing, previous)[0]
|
return self.classify(keys, finishing, previous)[0]
|
||||||
|
|
||||||
|
# --- segmentation ---
|
||||||
|
|
||||||
|
segment_penalty = 3.0
|
||||||
|
unconfirmed_switch_penalty = 2.0
|
||||||
|
|
||||||
|
def segment(self, keys, finishing=False, previous=None):
|
||||||
|
"""Splits keys into alternating segments: [(start, end, mode)], e.g.
|
||||||
|
"ji3hello" -> [(0, 3, zhuyin), (3, 8, literal)]. Dynamic programming over
|
||||||
|
split points; a Zhuyin segment followed by another must end with a tone."""
|
||||||
|
keys = list(keys)
|
||||||
|
n = len(keys)
|
||||||
|
if n == 0:
|
||||||
|
return []
|
||||||
|
modes = (ZHUYIN, LITERAL)
|
||||||
|
best = [[NEG_INF, NEG_INF] for _ in range(n + 1)]
|
||||||
|
back = [[-1, -1] for _ in range(n + 1)]
|
||||||
|
for i in range(1, n + 1):
|
||||||
|
for j in range(i):
|
||||||
|
scores = self._segment_scores(keys[j:i], i == n, j > 0, finishing,
|
||||||
|
previous if j == 0 else None, j == 0 and i == n)
|
||||||
|
for m in (0, 1):
|
||||||
|
if scores[m] == NEG_INF:
|
||||||
|
continue
|
||||||
|
if j == 0:
|
||||||
|
total = scores[m]
|
||||||
|
else:
|
||||||
|
if best[j][1 - m] == NEG_INF:
|
||||||
|
continue
|
||||||
|
total = best[j][1 - m] + scores[m] - self.segment_penalty
|
||||||
|
if total > best[i][m]:
|
||||||
|
best[i][m] = total
|
||||||
|
back[i][m] = j
|
||||||
|
if best[n][0] == NEG_INF and best[n][1] == NEG_INF:
|
||||||
|
return [(0, n, self.mode(keys, finishing, previous))]
|
||||||
|
m = 0 if best[n][0] >= best[n][1] else 1
|
||||||
|
out, i = [], n
|
||||||
|
while i > 0:
|
||||||
|
j = back[i][m]
|
||||||
|
out.append((j, i, modes[m]))
|
||||||
|
i, m = j, 1 - m
|
||||||
|
return out[::-1]
|
||||||
|
|
||||||
|
def _segment_scores(self, seg, is_last, after_other, finishing, previous, whole):
|
||||||
|
parse = bopomofo.parse(seg)
|
||||||
|
if not is_last and parse.pending is not None:
|
||||||
|
zh = NEG_INF
|
||||||
|
else:
|
||||||
|
zh = self.zhuyin_score(seg + [" "] if finishing and is_last else seg) + self.zhuyin_prior
|
||||||
|
if is_last and after_other and not finishing and not any(s.toned for s in parse.syllables):
|
||||||
|
zh -= self.unconfirmed_switch_penalty
|
||||||
|
lit = self.literal_score(seg, complete=(not is_last) or finishing)
|
||||||
|
if previous == ZHUYIN:
|
||||||
|
zh += self.context_weight
|
||||||
|
elif previous == LITERAL:
|
||||||
|
lit += self.context_weight
|
||||||
|
if whole:
|
||||||
|
b = self.learned.bias("".join(seg))
|
||||||
|
if b > 0:
|
||||||
|
zh += b
|
||||||
|
else:
|
||||||
|
lit -= b
|
||||||
|
return zh, lit
|
||||||
|
|
||||||
def zhuyin_score(self, keys):
|
def zhuyin_score(self, keys):
|
||||||
p = bopomofo.parse(keys)
|
p = bopomofo.parse(keys)
|
||||||
if p.foreign_keys > 0 or not p.syllables:
|
if p.foreign_keys > 0 or not p.syllables:
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
Keys are given as one-character strings, or one of the names in SPECIAL_KEYS.
|
Keys are given as one-character strings, or one of the names in SPECIAL_KEYS.
|
||||||
handle() returns (handled, commit_text_or_None).
|
handle() returns (handled, commit_text_or_None).
|
||||||
|
The buffer is split into Zhuyin / literal segments, e.g. "ji3hello" -> 我 + hello.
|
||||||
"""
|
"""
|
||||||
import time
|
import time
|
||||||
|
|
||||||
@@ -32,11 +33,14 @@ class CompositionEngine:
|
|||||||
self.now = now
|
self.now = now
|
||||||
self.keys = []
|
self.keys = []
|
||||||
self.forced_mode = None
|
self.forced_mode = None
|
||||||
self.fixed = {}
|
self.fixed = {} # zhuyin segment start -> {(syl_start, syl_end): text}
|
||||||
self.candidates = []
|
self.candidates = []
|
||||||
self.candidate_index = 0
|
self.candidate_index = 0
|
||||||
|
self._candidate_segment_start = 0
|
||||||
self._previous_mode = None
|
self._previous_mode = None
|
||||||
self._previous_time = float("-inf")
|
self._previous_time = float("-inf")
|
||||||
|
self._cache_key = None
|
||||||
|
self._cache_value = None
|
||||||
|
|
||||||
# --- derived state ---
|
# --- derived state ---
|
||||||
|
|
||||||
@@ -44,9 +48,26 @@ class CompositionEngine:
|
|||||||
def context(self):
|
def context(self):
|
||||||
return self._previous_mode if self.now() - self._previous_time < self.context_window else None
|
return self._previous_mode if self.now() - self._previous_time < self.context_window else None
|
||||||
|
|
||||||
|
def _segments(self, keys, finishing=False):
|
||||||
|
if not keys:
|
||||||
|
return []
|
||||||
|
if self.forced_mode:
|
||||||
|
return [(0, len(keys), self.forced_mode)]
|
||||||
|
ctx = self.context
|
||||||
|
cache_key = (tuple(keys), finishing, ctx)
|
||||||
|
if cache_key != self._cache_key:
|
||||||
|
self._cache_key = cache_key
|
||||||
|
self._cache_value = self.classifier.segment(keys, finishing, ctx)
|
||||||
|
return self._cache_value
|
||||||
|
|
||||||
|
@property
|
||||||
|
def segments(self):
|
||||||
|
return self._segments(self.keys)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def mode(self):
|
def mode(self):
|
||||||
return self.forced_mode or self.classifier.mode(self.keys, previous=self.context)
|
segs = self.segments
|
||||||
|
return segs[-1][2] if segs else ZHUYIN
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def showing_candidates(self):
|
def showing_candidates(self):
|
||||||
@@ -56,28 +77,28 @@ class CompositionEngine:
|
|||||||
def page_start(self):
|
def page_start(self):
|
||||||
return (self.candidate_index // self.page_size) * self.page_size
|
return (self.candidate_index // self.page_size) * self.page_size
|
||||||
|
|
||||||
def _parse(self):
|
def _render(self, seg, final):
|
||||||
return bopomofo.parse(self.keys)
|
start, end, mode = seg
|
||||||
|
seg_keys = self.keys[start:end]
|
||||||
|
if mode == LITERAL:
|
||||||
|
return "".join(seg_keys)
|
||||||
|
p = bopomofo.parse(seg_keys)
|
||||||
|
fixed_here = self.fixed.get(start, {})
|
||||||
|
syl = [s.syllable for s in (p.syllables if final else p.completed)]
|
||||||
|
fx = {k: v for k, v in fixed_here.items() if k[1] <= len(syl)}
|
||||||
|
text = "".join(t for _, t in self.converter.convert(syl, fx))
|
||||||
|
if not final and p.pending:
|
||||||
|
text += p.pending.syllable.reading
|
||||||
|
return text
|
||||||
|
|
||||||
def _valid_fixed(self, p):
|
@property
|
||||||
n = len(p.completed)
|
def marked_pieces(self):
|
||||||
return {k: v for k, v in self.fixed.items() if k[1] <= n}
|
segs = self.segments
|
||||||
|
return [(self._render(s, i < len(segs) - 1), s[2]) for i, s in enumerate(segs)]
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def marked_text(self):
|
def marked_text(self):
|
||||||
if not self.keys:
|
return "".join(t for t, _ in self.marked_pieces)
|
||||||
return ""
|
|
||||||
if self.mode == LITERAL:
|
|
||||||
return "".join(self.keys)
|
|
||||||
p = self._parse()
|
|
||||||
done = self.converter.convert([s.syllable for s in p.completed], self._valid_fixed(p))
|
|
||||||
pending = p.pending.syllable.reading if p.pending else ""
|
|
||||||
return "".join(t for _, t in done) + pending
|
|
||||||
|
|
||||||
def _zhuyin_commit_text(self):
|
|
||||||
p = self._parse()
|
|
||||||
segs = self.converter.convert([s.syllable for s in p.syllables], self._valid_fixed(p))
|
|
||||||
return "".join(t for _, t in segs)
|
|
||||||
|
|
||||||
# --- key handling ---
|
# --- key handling ---
|
||||||
|
|
||||||
@@ -102,7 +123,7 @@ class CompositionEngine:
|
|||||||
if not self.keys:
|
if not self.keys:
|
||||||
return PASS
|
return PASS
|
||||||
self.keys.pop()
|
self.keys.pop()
|
||||||
self.fixed = self._valid_fixed(self._parse())
|
self.fixed = {k: v for k, v in self.fixed.items() if k < len(self.keys)}
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
self.forced_mode = None
|
self.forced_mode = None
|
||||||
self.fixed = {}
|
self.fixed = {}
|
||||||
@@ -128,25 +149,32 @@ class CompositionEngine:
|
|||||||
return CONSUMED if self.keys else PASS
|
return CONSUMED if self.keys else PASS
|
||||||
return self._handle_character(key)
|
return self._handle_character(key)
|
||||||
|
|
||||||
|
def _last_parse(self, segs):
|
||||||
|
if not segs or segs[-1][2] != ZHUYIN:
|
||||||
|
return None
|
||||||
|
start, end, _ = segs[-1]
|
||||||
|
return bopomofo.parse(self.keys[start:end])
|
||||||
|
|
||||||
def _handle_character(self, c):
|
def _handle_character(self, c):
|
||||||
if c == " ":
|
if c == " ":
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
return PASS
|
return PASS
|
||||||
decided = self.forced_mode or self.classifier.mode(self.keys, finishing=True, previous=self.context)
|
segs = self._segments(self.keys, finishing=True)
|
||||||
if decided == LITERAL:
|
if segs[-1][2] == LITERAL:
|
||||||
return True, self._commit(LITERAL) + " "
|
return True, self._commit(segs) + " "
|
||||||
if self._parse().pending is not None:
|
p = self._last_parse(segs)
|
||||||
|
if p is not None and p.pending is not None:
|
||||||
self.keys.append(" ")
|
self.keys.append(" ")
|
||||||
return CONSUMED
|
return CONSUMED
|
||||||
return True, self._commit(ZHUYIN)
|
return True, self._commit(segs)
|
||||||
|
|
||||||
composable = bopomofo.is_zhuyin_key(c) or (c.isascii() and c.isalpha()) or (c == "'" and self.keys)
|
composable = bopomofo.is_zhuyin_key(c) or (c.isascii() and c.isalpha()) or (c == "'" and self.keys)
|
||||||
if composable:
|
if composable:
|
||||||
# ", . / ; -" are Zhuyin symbols but act as punctuation after an English word.
|
# ", . / ; -" are Zhuyin symbols but act as punctuation after an English word.
|
||||||
if self.keys and c in bopomofo.SYMBOLS and not c.isalnum() and self.forced_mode != ZHUYIN:
|
if self.keys and c in bopomofo.SYMBOLS and not c.isalnum() and self.forced_mode != ZHUYIN:
|
||||||
with_key = self.forced_mode or self.classifier.mode(self.keys + [c], previous=self.context)
|
with_key = self._segments(self.keys + [c])
|
||||||
if with_key == LITERAL and not all(k.isdigit() for k in self.keys):
|
if with_key[-1][2] == LITERAL and not all(k.isdigit() for k in self.keys):
|
||||||
return True, self._commit(LITERAL) + c
|
return True, self._commit() + c
|
||||||
self.keys.append(c)
|
self.keys.append(c)
|
||||||
return CONSUMED
|
return CONSUMED
|
||||||
|
|
||||||
@@ -157,14 +185,18 @@ class CompositionEngine:
|
|||||||
return PASS
|
return PASS
|
||||||
return True, committed + punct
|
return True, committed + punct
|
||||||
|
|
||||||
# --- candidates ---
|
# --- candidates (for the last Zhuyin segment) ---
|
||||||
|
|
||||||
def _open_candidates(self):
|
def _open_candidates(self):
|
||||||
p = self._parse()
|
segs = self.segments
|
||||||
if self.mode != ZHUYIN or not p.completed:
|
if not segs or segs[-1][2] != ZHUYIN:
|
||||||
|
return
|
||||||
|
start, end, _ = segs[-1]
|
||||||
|
syl = [s.syllable for s in bopomofo.parse(self.keys[start:end]).completed]
|
||||||
|
if not syl:
|
||||||
return
|
return
|
||||||
syl = [s.syllable for s in p.completed]
|
|
||||||
self.candidates = self.converter.candidates(syl, len(syl))
|
self.candidates = self.converter.candidates(syl, len(syl))
|
||||||
|
self._candidate_segment_start = start
|
||||||
self.candidate_index = 0
|
self.candidate_index = 0
|
||||||
|
|
||||||
def page(self):
|
def page(self):
|
||||||
@@ -176,8 +208,10 @@ class CompositionEngine:
|
|||||||
if not 0 <= index < len(self.candidates):
|
if not 0 <= index < len(self.candidates):
|
||||||
return
|
return
|
||||||
rng, text = self.candidates[index]
|
rng, text = self.candidates[index]
|
||||||
self.fixed = {k: v for k, v in self.fixed.items() if not (k[0] < rng[1] and rng[0] < k[1])}
|
here = {k: v for k, v in self.fixed.get(self._candidate_segment_start, {}).items()
|
||||||
self.fixed[rng] = text
|
if not (k[0] < rng[1] and rng[0] < k[1])}
|
||||||
|
here[rng] = text
|
||||||
|
self.fixed[self._candidate_segment_start] = here
|
||||||
self.candidates = []
|
self.candidates = []
|
||||||
|
|
||||||
def _handle_candidate_key(self, key):
|
def _handle_candidate_key(self, key):
|
||||||
@@ -195,12 +229,12 @@ class CompositionEngine:
|
|||||||
|
|
||||||
# --- commit ---
|
# --- commit ---
|
||||||
|
|
||||||
def _commit(self, decided=None):
|
def _commit(self, segs=None):
|
||||||
m = decided or self.mode
|
segs = segs if segs is not None else self.segments
|
||||||
text = "".join(self.keys) if m == LITERAL else self._zhuyin_commit_text()
|
text = "".join(self._render(s, True) for s in segs)
|
||||||
if self.forced_mode:
|
if self.forced_mode:
|
||||||
self.classifier.learned.record("".join(self.keys), self.forced_mode)
|
self.classifier.learned.record("".join(self.keys), self.forced_mode)
|
||||||
self._previous_mode = m
|
self._previous_mode = segs[-1][2] if segs else None
|
||||||
self._previous_time = self.now()
|
self._previous_time = self.now()
|
||||||
self.reset()
|
self.reset()
|
||||||
return text
|
return text
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "自動注音 AutoZhuyin",
|
"name": "自動注音 AutoZhuyin",
|
||||||
"version": "0.1.0",
|
"version": "0.2.0",
|
||||||
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
||||||
"locale": "zh-Hant-TW",
|
"locale": "zh-Hant-TW",
|
||||||
"fallbackLocale": "zh-TW",
|
"fallbackLocale": "zh-TW",
|
||||||
|
|||||||
@@ -154,5 +154,39 @@ class EngineTests(unittest.TestCase):
|
|||||||
self.assertEqual(type_keys("uvl3\n", make_engine()), "小")
|
self.assertEqual(type_keys("uvl3\n", make_engine()), "小")
|
||||||
|
|
||||||
|
|
||||||
|
class MixedSegmentTests(unittest.TestCase):
|
||||||
|
def run_keys(self, s):
|
||||||
|
e = make_engine()
|
||||||
|
return type_keys(s, e), e.marked_text
|
||||||
|
|
||||||
|
def test_chinese_then_english(self):
|
||||||
|
self.assertEqual(self.run_keys("ji3hello ")[0], "我hello ")
|
||||||
|
self.assertEqual(self.run_keys("su3cl3github\n")[0], "你好github")
|
||||||
|
self.assertEqual(self.run_keys("ji3hel")[1], "我hel")
|
||||||
|
|
||||||
|
def test_english_then_chinese(self):
|
||||||
|
self.assertEqual(self.run_keys("hellosu3cl3 ")[0], "hello你好")
|
||||||
|
self.assertEqual(self.run_keys("oksu3cl3\n")[0], "ok你好")
|
||||||
|
|
||||||
|
def test_three_segments(self):
|
||||||
|
self.assertEqual(self.run_keys("ji3emailsu3cl3\n")[0], "我email你好")
|
||||||
|
|
||||||
|
def test_single_language_unchanged(self):
|
||||||
|
self.assertEqual(self.run_keys("rup wu0 wu0 fu4cp3cl3\n")[0], "今天天氣很好")
|
||||||
|
self.assertEqual(self.run_keys("helloworld\n")[0], "helloworld")
|
||||||
|
self.assertEqual(self.run_keys("meeting ")[0], "meeting ")
|
||||||
|
self.assertEqual(len(self.run_keys("183\n")[0]), 1)
|
||||||
|
|
||||||
|
def test_speed(self):
|
||||||
|
e = make_engine()
|
||||||
|
start = time.time()
|
||||||
|
for c in "ji3emailsu3cl3rup wu0 wu0 fu4":
|
||||||
|
e.handle(c)
|
||||||
|
e.marked_text
|
||||||
|
per_key = (time.time() - start) / 30
|
||||||
|
print("\nper key: %.1f ms" % (per_key * 1000))
|
||||||
|
self.assertLess(per_key, 0.05)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
Reference in New Issue
Block a user