Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9cac3467a0 | ||
|
|
49e0b52fd5 | ||
|
|
fcc0f9d2db | ||
|
|
31d9991e16 | ||
|
|
6b4ec39dc1 | ||
|
|
955b0ca5c3 | ||
|
|
251a16fa08 | ||
|
|
7cb1f62dfa | ||
|
|
3670305f0f | ||
|
|
ef9f91a6b7 | ||
|
|
1dacb8f164 | ||
|
|
bfeed22916 | ||
|
|
bb9cb0c4e0 | ||
|
|
2c5c70c066 | ||
|
|
17f7d6f50d | ||
|
|
7e173a1da8 |
@@ -1,6 +1,6 @@
|
||||
# 自動注音 AutoZhuyin
|
||||
|
||||
給繁體中文使用者的輸入法(macOS、Windows):**不用按切換鍵**,直接用大千注音鍵盤打字,輸入法會自動判斷你打的是注音還是英文。
|
||||
給繁體中文使用者的輸入法(macOS、Windows、Linux):**不用按切換鍵**,直接用大千注音鍵盤打字,輸入法會自動判斷你打的是注音還是英文。
|
||||
|
||||
```
|
||||
按鍵: ji3 hello world su3cl3␣
|
||||
@@ -12,11 +12,12 @@
|
||||
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
|
||||
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
|
||||
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
|
||||
- **單按 Shift 強制英文**:要打英文時先單按一下 Shift,接下來這個字照打、保持小寫,而且直接送出、不顯示底線
|
||||
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷
|
||||
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
|
||||
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
|
||||
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
|
||||
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`
|
||||
- **中間補字、刪字**:游標移到前面後直接打注音,會插在游標處;Backspace 刪掉游標前的那個字
|
||||
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`;任何時候按 `` ` `` 再按標點鍵,一定得到全形標點(例如 `` `< `` = `,`)
|
||||
- **完整詞庫**:約 17 萬筆,來自小麥注音 McBopomofo(MIT 授權)
|
||||
|
||||
## 下載與安裝
|
||||
@@ -44,6 +45,16 @@ PKG 會裝到 `/Library/Input Methods/`,系統上所有使用者都能用。
|
||||
|
||||
SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安裝:以系統管理員身分執行 `uninstall.bat`。
|
||||
|
||||
### Linux(Bazzite、SteamOS 桌面模式、Fedora…)
|
||||
|
||||
需求:系統內建 IBus 輸入法框架(Bazzite、Fedora 預設就有)。只在桌面模式可用,Steam 遊戲模式不支援系統輸入法。
|
||||
|
||||
1. 解壓縮 `AutoZhuyin-Linux-<版本>.tar.gz`,在終端機進到資料夾執行 `./install.sh`。不需要 sudo,只會裝在家目錄(`~/.local/share/autozhuyin`),所以唯讀系統也能用。
|
||||
2. KDE(Bazzite / SteamOS 預設):系統設定 → 鍵盤 → 虛擬鍵盤,選 **IBus Wayland** → 套用,登出再登入。GNOME 不用設定。
|
||||
3. 之後每次登入會自動切換到自動注音(`~/.config/autostart/autozhuyin.desktop`)。
|
||||
|
||||
暫時改回一般鍵盤:`ibus engine xkb:us::eng`;切回:`ibus engine autozhuyin`;移除:`./uninstall.sh`。
|
||||
|
||||
## 按鍵
|
||||
|
||||
| 按鍵 | 作用 |
|
||||
@@ -51,13 +62,14 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
| 空白 | 注音:一聲;音節已完成時送出中文。英文:送出單字加空白 |
|
||||
| Enter | 送出目前組字 |
|
||||
| **Tab** | 判斷錯了?按 Tab 把整段翻轉成另一種語言,輸入法會**記住**這次修正 |
|
||||
| ← → | 在組字區移動游標,選要改哪個字 |
|
||||
| ← → | 在組字區移動游標(可移到最前面)。游標在中間時:直接打注音會**插入**新字,Backspace 刪掉游標前的字,↓ 選游標前的字 |
|
||||
| ↓ | 開啟游標前那個字或詞的候選字(游標預設在最後),數字鍵 1–9 選字,Esc 關閉 |
|
||||
| Esc | 取消組字 |
|
||||
| **單按 Shift** | 沒在打字時:下一個字強制英文(保持小寫),字母直接送出、沒有底線;按空白、Enter 或其他符號後恢復自動判斷,再按一次 Shift 取消。打到一半時:同 Tab,整段翻轉 |
|
||||
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷。打到一半時:同 Tab,整段翻轉 |
|
||||
| Shift + 字母 | 打大寫字母(也會強制英文) |
|
||||
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
|
||||
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
|
||||
| `` ` `` + 標點鍵 | **強制全形標點**,不管前後是中文還是英文:`` `, `` `` `< `` = `,`、`` `. `` `` `> `` = `。`、`` `/ `` = `?`、`` `; `` = `;`、`` `: `` = `:`、`` `' `` `` `\ `` = `、`、`` `[ `` `` `] `` = `「」`、`` `{ `` `` `} `` = `『』`、`` `( `` `` `) `` = `()`、`` `! `` = `!`、`` `~ `` = `~`、`` `- `` = `——`、`` `= `` = `=`。`` ` `` 後面接其他鍵就是普通的反引號(` ``` ` 照常可用) |
|
||||
|
||||
## 運作方式
|
||||
|
||||
@@ -85,10 +97,10 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
|
||||
## 個人資料
|
||||
|
||||
| | macOS | Windows |
|
||||
|---|---|---|
|
||||
| 學習紀錄(Tab 修正) | `UserDefaults`(`com.autozhuyin.inputmethod.AutoZhuyin`) | `%APPDATA%\PIME\autozhuyin\learned.json` |
|
||||
| 自訂詞 | `~/Library/Application Support/AutoZhuyin/user-phrases.txt` | `%APPDATA%\PIME\autozhuyin\user-phrases.txt` |
|
||||
| | macOS | Windows | Linux |
|
||||
|---|---|---|---|
|
||||
| 學習紀錄(Tab 修正) | `UserDefaults`(`com.autozhuyin.inputmethod.AutoZhuyin`) | `%APPDATA%\PIME\autozhuyin\learned.json` | `~/.config/autozhuyin/learned.json` |
|
||||
| 自訂詞 | `~/Library/Application Support/AutoZhuyin/user-phrases.txt` | `%APPDATA%\PIME\autozhuyin\user-phrases.txt` | `~/.config/autozhuyin/user-phrases.txt` |
|
||||
|
||||
自訂詞的格式和 `lexicon.txt` 一樣,每行一個詞:`ㄗˋ-ㄉㄨㄥˋ-ㄓㄨˋ-ㄧㄣ 自動注音 -4`(讀音、詞、分數;分數越大越優先)。
|
||||
Mac 版可以在輸入法選單選「清除學習紀錄」。
|
||||
@@ -124,6 +136,18 @@ scripts/package-windows.sh # 產生 build/AutoZhuyin-Wi
|
||||
- `windows/autozhuyin/data/` 是產生出來的,不進版本控制。
|
||||
- 版本號在 `windows/autozhuyin/ime.json`。
|
||||
|
||||
### Linux
|
||||
|
||||
Linux 版是 IBus 輸入法(`linux/autozhuyin_ibus.py`),直接使用 Windows 版的 Python 核心(`windows/autozhuyin/core/`)和資料。
|
||||
|
||||
```bash
|
||||
python3 -m unittest discover -s linux/tests # 用假的 IBus 模組測試外殼
|
||||
scripts/package-linux.sh # 產生 build/AutoZhuyin-Linux-<版本>.tar.gz(會先跑測試)
|
||||
```
|
||||
|
||||
- 安裝在使用者目錄,不寫入 `/usr`:開機時由 autostart 啟動 `autozhuyin_ibus.py --activate`,它會把引擎動態註冊給 ibus-daemon 並切換過去。
|
||||
- 也支援由 ibus-daemon 啟動(`--ibus`,配合 `/usr/share/ibus/component` 的 XML),之後要做 RPM 時可以用。
|
||||
|
||||
> 修改核心邏輯時,Swift(`Sources/AutoZhuyinCore/`)和 Python(`windows/autozhuyin/core/`)兩邊都要改,並在兩邊的測試加上相同案例。
|
||||
|
||||
### 訓練判斷權重
|
||||
@@ -144,12 +168,12 @@ python3 training/make_parity.py # 權重或特徵改了之後都要重
|
||||
|
||||
目前的結果(10,000 筆測試資料):
|
||||
|
||||
| | 送出時整段判斷正確 | 打字過程中顯示正確 |
|
||||
|---|---|---|
|
||||
| 手調權重 | 97.58% | 86.15% |
|
||||
| 訓練後 | **99.62%** | 87.81% |
|
||||
| | 送出時整段判斷正確 | 打字過程中顯示正確 | 跟前一個字語言不同時 |
|
||||
|---|---|---|---|
|
||||
| 手調權重 | 97.84% | 88.19% | 96.2% |
|
||||
| 訓練後 | **99.52%** | **91.49%** | **99.3%** |
|
||||
|
||||
送出時的錯誤率從 2.4% 降到 0.4%。「打字過程中」本來就有歧義(例如 `ji3h` 的 `h` 可能是英文的開頭,也可能是注音 ㄘ),所以準確率比較低,但最後送出的結果才是重點。
|
||||
訓練資料也模擬了「前一個字是中文或英文」和「按下空白(一聲)的那一刻」,所以「前一個字」的影響(`ctx.zh` / `ctx.lit`)也是訓練出來的,不是手調的。「打字過程中」本來就有歧義(例如 `ji3h` 的 `h` 可能是英文的開頭,也可能是注音 ㄘ),所以準確率比較低,但最後送出的結果才是重點。
|
||||
|
||||
新增特徵時,Swift 和 Python 兩邊都要加,預設權重設成 0,然後重新訓練。
|
||||
|
||||
@@ -177,12 +201,12 @@ scripts/release.sh minor --push # 同時 push commit 和 tag
|
||||
`release.sh` 會自動:
|
||||
1. 遞增版本號(`Resources/Info.plist` 的版本與 build 號、`windows/autozhuyin/ime.json`)
|
||||
2. 跑 Swift 和 Python 全部測試
|
||||
3. 產生 `build/AutoZhuyin-<版本>.pkg` 和 `build/AutoZhuyin-Windows-<版本>.zip`
|
||||
3. 產生 `build/AutoZhuyin-<版本>.pkg`、`build/AutoZhuyin-Windows-<版本>.zip`、`build/AutoZhuyin-Linux-<版本>.tar.gz`
|
||||
4. commit「Release v<版本>」並打上 `v<版本>` tag
|
||||
|
||||
任何一步失敗都會把版本號改回去。只想改版本號不打包:`scripts/bump-version.sh [patch|minor|major]`。
|
||||
|
||||
最後在 Gitea 用這個 tag 建立 Release,附上 `build/` 裡的 `.pkg` 和 `.zip`。
|
||||
最後在 Gitea 用這個 tag 建立 Release,附上 `build/` 裡的 `.pkg`、`.zip` 和 `.tar.gz`。
|
||||
|
||||
## 專案結構
|
||||
|
||||
@@ -210,14 +234,20 @@ windows/
|
||||
installer/ install.bat、uninstall.bat、給使用者的 README.txt
|
||||
tests/ Python 測試
|
||||
tools/ 產生資料檔、圖示
|
||||
linux/
|
||||
autozhuyin_ibus.py IBus 外殼(共用 windows/autozhuyin/core)
|
||||
installer/ install.sh、uninstall.sh、給使用者的 README.txt
|
||||
tests/ 用假 IBus 模組的外殼測試
|
||||
training/ 合成資料、訓練權重、Swift/Python 一致性測試資料
|
||||
THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝檔)
|
||||
```
|
||||
|
||||
## 已知限制
|
||||
|
||||
- Windows 版還沒在實機上測試過。
|
||||
- Windows 版、Linux 版還沒在實機上測試過。
|
||||
- Linux 版只支援 IBus。KDE 上有些 Flatpak 程式(例如 Flatpak 版 Chrome)在 IBus Wayland 下可能打不出字。
|
||||
- 聲調鍵打在注音符號中間(例如 `uv3l`)無法自動修正。
|
||||
- 很短的字串(如 `up` 是英文還是 ㄧㄣ)本質上就有歧義,可能需要按 Tab 修正;修正後會記住。
|
||||
- 很短的字串本質上就有歧義。像 `up`(也是 ㄧㄣ 因)這種常用的,按空白後會先保留,看下一個詞再決定(`up jo4` = 因為、`up the` = up the);其他判斷錯的可以按 Tab 修正,修正後會記住。
|
||||
- 安裝檔沒有簽章(macOS 沒有 Developer ID,Windows 沒有程式碼簽章憑證)。
|
||||
|
||||
## 下一步(Roadmap)
|
||||
@@ -225,6 +255,8 @@ training/ 合成資料、訓練權重、Swift/Python 一致
|
||||
- [x] 匯入完整詞庫
|
||||
- [x] 注音順序錯誤修正
|
||||
- [x] Windows 版
|
||||
- [x] Linux 版(IBus)
|
||||
- [ ] Linux Fcitx5 版(KDE 上相容性較好)
|
||||
- [x] 用資料訓練判斷權重(合成資料)
|
||||
- [ ] 加入真實打字資料一起訓練
|
||||
- [ ] 已送出的英文單字若判斷錯誤,可以「事後翻轉」
|
||||
@@ -237,3 +269,5 @@ training/ 合成資料、訓練權重、Swift/Python 一致
|
||||
- 詞庫資料:[小麥注音 McBopomofo](https://github.com/openvanilla/McBopomofo)(MIT License)
|
||||
- Windows 輸入法框架:[PIME](https://github.com/EasyIME/PIME)(LGPL 2.1,使用者自行安裝,本專案不包含其程式碼)
|
||||
- 英文字典:Webster's Second International Dictionary(公有領域,macOS 的 `/usr/share/dict/words`)
|
||||
|
||||
完整的第三方授權聲明(含小麥注音 MIT 授權全文)見 [THIRD_PARTY_LICENSES.txt](THIRD_PARTY_LICENSES.txt),Mac 版放在 `AutoZhuyin.app/Contents/Resources/`,Windows 版放在 zip 根目錄和安裝後的 `autozhuyin` 資料夾。
|
||||
|
||||
@@ -15,9 +15,9 @@
|
||||
<key>CFBundlePackageType</key>
|
||||
<string>APPL</string>
|
||||
<key>CFBundleShortVersionString</key>
|
||||
<string>0.5.1</string>
|
||||
<string>0.9.0</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>7</string>
|
||||
<string>15</string>
|
||||
<key>InputMethodConnectionName</key>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
||||
<key>InputMethodServerControllerClass</key>
|
||||
|
||||
@@ -1,39 +1,41 @@
|
||||
{
|
||||
"about": "AutoZhuyin classifier weights, learned by training/train.py",
|
||||
"trained": "2026-11-11",
|
||||
"examples": 40001,
|
||||
"examples": 40000,
|
||||
"test_accuracy": {
|
||||
"commit": 0.9962,
|
||||
"typing": 0.8781
|
||||
"commit": 0.9952,
|
||||
"typing": 0.9149
|
||||
},
|
||||
"weights": {
|
||||
"lit.bias": 0.4276,
|
||||
"lit.common": -0.3465,
|
||||
"lit.common_len": 0.81,
|
||||
"lit.dict": -1.7902,
|
||||
"lit.dict_len": 0.6765,
|
||||
"lit.digits": -2.7403,
|
||||
"lit.num_digits": -0.0751,
|
||||
"lit.num_others": -0.7433,
|
||||
"lit.others": -1.5685,
|
||||
"lit.prefix_common_len": 0.2082,
|
||||
"lit.prefix_dict_len": 0.0003,
|
||||
"lit.short": -0.2397,
|
||||
"lit.unknown_score": 0.213,
|
||||
"ctx.lit": 0.0916,
|
||||
"ctx.zh": 0.1349,
|
||||
"lit.bias": 0.3982,
|
||||
"lit.common": -0.2906,
|
||||
"lit.common_len": 0.8046,
|
||||
"lit.dict": -1.6662,
|
||||
"lit.dict_len": 0.6441,
|
||||
"lit.digits": -2.9689,
|
||||
"lit.num_digits": -0.018,
|
||||
"lit.num_others": -0.5554,
|
||||
"lit.others": -2.0002,
|
||||
"lit.prefix_common_len": 0.2831,
|
||||
"lit.prefix_dict_len": 0.0299,
|
||||
"lit.short": -0.2482,
|
||||
"lit.unknown_score": 0.2479,
|
||||
"lit.upper": 20.0,
|
||||
"seg.switch": -1.7203,
|
||||
"zh.bias": 1.3522,
|
||||
"zh.lexscore": 0.26,
|
||||
"seg.switch": -1.7596,
|
||||
"zh.bias": 1.3423,
|
||||
"zh.lexscore": 0.2917,
|
||||
"zh.pending_invalid": -5.0,
|
||||
"zh.pending_prefix": -0.9295,
|
||||
"zh.phrase": 0.4595,
|
||||
"zh.reordered": -1.4348,
|
||||
"zh.pending_prefix": -0.9162,
|
||||
"zh.phrase": 0.4499,
|
||||
"zh.reordered": -1.3079,
|
||||
"zh.stray": -5.0,
|
||||
"zh.toned_invalid": -5.0,
|
||||
"zh.toned_valid_lex": -0.9823,
|
||||
"zh.toned_valid_nolex": -1.8742,
|
||||
"zh.unconfirmed": -0.7891,
|
||||
"zh.toned_valid_lex": -0.9018,
|
||||
"zh.toned_valid_nolex": -1.6341,
|
||||
"zh.unconfirmed": -0.7952,
|
||||
"zh.untoned_mid_invalid": -5.0,
|
||||
"zh.untoned_mid_valid": -2.8009
|
||||
"zh.untoned_mid_valid": -2.7479
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,22 +11,28 @@ enum Shared {
|
||||
|
||||
static let classifier: Classifier = {
|
||||
let bundle = Bundle.main
|
||||
let start = Date()
|
||||
let lexicon = Lexicon()
|
||||
if let url = bundle.url(forResource: "lexicon", withExtension: "txt"),
|
||||
let text = try? String(contentsOf: url, encoding: .utf8) {
|
||||
lexicon.load(text)
|
||||
let data = try? Data(contentsOf: url) {
|
||||
lexicon.load(data: data)
|
||||
}
|
||||
// Optional user phrases: ~/Library/Application Support/AutoZhuyin/user-phrases.txt
|
||||
if let support = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first {
|
||||
let userFile = support.appendingPathComponent("AutoZhuyin/user-phrases.txt")
|
||||
if let text = try? String(contentsOf: userFile, encoding: .utf8) { lexicon.load(text) }
|
||||
}
|
||||
let english = EnglishModel(commonWordsURL: bundle.url(forResource: "english-common", withExtension: "txt"))
|
||||
let english = EnglishModel(commonWordsURL: bundle.url(forResource: "english-common", withExtension: "txt"),
|
||||
dictionaryURL: bundle.url(forResource: "english-dict", withExtension: "txt"),
|
||||
bigramURL: bundle.url(forResource: "english-bigram", withExtension: "json"))
|
||||
let saved = UserDefaults.standard.dictionary(forKey: learnedKey) as? [String: Int] ?? [:]
|
||||
let learned = LearnedPreferences(weights: saved) { weights in
|
||||
UserDefaults.standard.set(weights, forKey: learnedKey)
|
||||
}
|
||||
let trained = Features.loadWeights(from: bundle.url(forResource: "classifier-weights", withExtension: "json"))
|
||||
NSLog("AutoZhuyin: data loaded in %.0f ms", Date().timeIntervalSince(start) * 1000)
|
||||
// Only needed for untoned syllables; build it off the startup path.
|
||||
DispatchQueue.global(qos: .utility).async { lexicon.prepareTonelessIndex() }
|
||||
return Classifier(lexicon: lexicon, english: english, learned: learned, weights: trained)
|
||||
}()
|
||||
}
|
||||
|
||||
@@ -58,9 +58,6 @@ public final class Classifier {
|
||||
/// Feature weights: the hand-tuned defaults overlaid with trained ones.
|
||||
public private(set) var weights: FeatureVector
|
||||
|
||||
/// How much the previous segment's mode pulls the current one.
|
||||
public var contextWeight = 1.5
|
||||
|
||||
public init(lexicon: Lexicon, english: EnglishModel, learned: LearnedPreferences = LearnedPreferences(),
|
||||
weights: FeatureVector = [:]) {
|
||||
self.lexicon = lexicon
|
||||
@@ -69,23 +66,22 @@ public final class Classifier {
|
||||
self.weights = Features.defaultWeights.merging(weights) { _, new in new }
|
||||
}
|
||||
|
||||
/// (Zhuyin features or nil, literal features or nil) for one segment.
|
||||
public func segmentFeatures(_ seg: [Character], isLast: Bool, afterOther: Bool,
|
||||
finishing: Bool) -> (FeatureVector?, FeatureVector?) {
|
||||
(Features.zhuyin(seg, lexicon: lexicon, isLast: isLast, afterOther: afterOther, finishing: finishing),
|
||||
Features.literal(seg, english: english, isLast: isLast, afterOther: afterOther, finishing: finishing))
|
||||
/// (Zhuyin features or nil, literal features or nil) for one segment. `previous` (mode of
|
||||
/// the text committed just before) only affects the first segment.
|
||||
public func segmentFeatures(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
|
||||
previous: InputMode? = nil) -> (FeatureVector?, FeatureVector?) {
|
||||
(Features.zhuyin(seg, lexicon: lexicon, isLast: isLast, afterOther: afterOther,
|
||||
finishing: finishing, previous: previous),
|
||||
Features.literal(seg, english: english, isLast: isLast, afterOther: afterOther,
|
||||
finishing: finishing, previous: previous))
|
||||
}
|
||||
|
||||
private func scores(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
|
||||
previous: InputMode?, whole: Bool) -> [Double] {
|
||||
let (fz, fl) = segmentFeatures(seg, isLast: isLast, afterOther: afterOther, finishing: finishing)
|
||||
let (fz, fl) = segmentFeatures(seg, isLast: isLast, afterOther: afterOther,
|
||||
finishing: finishing, previous: previous)
|
||||
var zh = fz.map { Features.dot(weights, $0) } ?? -.infinity
|
||||
var lit = fl.map { Features.dot(weights, $0) } ?? -.infinity
|
||||
switch previous {
|
||||
case .zhuyin?: zh += contextWeight
|
||||
case .literal?: lit += contextWeight
|
||||
case nil: break
|
||||
}
|
||||
if whole {
|
||||
let b = learned.bias(for: String(seg))
|
||||
if b > 0 { zh += b } else { lit -= b }
|
||||
|
||||
@@ -35,22 +35,33 @@ public final class CompositionEngine {
|
||||
public let converter: Converter
|
||||
|
||||
public private(set) var keys: [Character] = []
|
||||
/// Set by Tab / Shift tap: overrides the classifier for the whole buffer.
|
||||
/// Set by Tab / a Shift tap while typing: overrides the classifier for the whole
|
||||
/// buffer. It is a correction, learned on commit.
|
||||
public private(set) var forcedMode: InputMode?
|
||||
/// True when forcedMode corrects the classifier (learned on commit); false when the
|
||||
/// user forced English before typing (Shift tap on an empty buffer), which is not a
|
||||
/// correction and is not learned.
|
||||
private var forcedIsCorrection = false
|
||||
/// English mode, toggled by a Shift tap when nothing is being typed. Keys go straight
|
||||
/// to the app (no composition, no underline) until Shift is tapped again.
|
||||
public private(set) var isEnglishMode = false
|
||||
/// User-picked phrases: Zhuyin segment start (key index) → syllable range → text.
|
||||
private var fixed: [Int: [Range<Int>: String]] = [:]
|
||||
public private(set) var candidates: [Candidate] = []
|
||||
private var candidateSegmentStart = 0
|
||||
/// Composition cursor for choosing candidates: index into `cursorStops`, nil = at the end.
|
||||
/// ← → move it; ↓ shows candidates for the phrase ending at the cursor.
|
||||
/// Composition cursor: a key index into `keys` (nil = at the end). ← → move it between
|
||||
/// syllables; ↓ shows candidates for the phrase before it; typing there inserts a
|
||||
/// syllable (kept in `inserting` until its tone key, then spliced into `keys`).
|
||||
private var cursor: Int?
|
||||
private var inserting: [Character] = []
|
||||
public var candidateIndex = 0
|
||||
public var isShowingCandidates: Bool { !candidates.isEmpty }
|
||||
|
||||
/// Index of a held space: an English word that is also a common one-syllable Chinese
|
||||
/// reading ("up " = ㄧㄣ 因) is not committed on space; the next word decides
|
||||
/// ("up jo4" = 因為, "up the" = up the).
|
||||
private var held: Int?
|
||||
/// "`" was typed: if the next key is in `symbols` it gives that full-width symbol
|
||||
/// whatever the context; anything else makes it a plain backtick (so ``` still works).
|
||||
public private(set) var pendingBacktick = false
|
||||
static let holdMinScore = -3.5
|
||||
|
||||
private var previousMode: InputMode?
|
||||
private var previousCommitTime = Date.distantPast
|
||||
/// How long the previous segment's mode keeps influencing the next one.
|
||||
@@ -71,9 +82,33 @@ public final class CompositionEngine {
|
||||
private func segments(of keys: [Character], finishing: Bool = false) -> [Segment] {
|
||||
guard !keys.isEmpty else { return [] }
|
||||
if let forced = forcedMode { return [Segment(range: 0..<keys.count, mode: forced)] }
|
||||
if let h = held, h < keys.count, keys[h] == " " {
|
||||
return heldSegments(keys, held: h, finishing: finishing)
|
||||
}
|
||||
return classifier.segment(keys, finishing: finishing, previous: context)
|
||||
}
|
||||
|
||||
/// The held word is Chinese if the words typed after it start with Chinese.
|
||||
private func heldSegments(_ keys: [Character], held h: Int, finishing: Bool) -> [Segment] {
|
||||
let base = h + 1
|
||||
let rest = classifier.segment(Array(keys[base...]), finishing: finishing, previous: nil).map {
|
||||
Segment(range: ($0.range.lowerBound + base)..<($0.range.upperBound + base), mode: $0.mode)
|
||||
}
|
||||
if let first = rest.first, first.mode == .zhuyin {
|
||||
return [Segment(range: 0..<first.range.upperBound, mode: .zhuyin)] + rest.dropFirst()
|
||||
}
|
||||
return [Segment(range: 0..<base, mode: .literal)] + rest
|
||||
}
|
||||
|
||||
/// Whether keys + space also read as a common Chinese syllable (see `held`).
|
||||
private func isAmbiguousWord(_ keys: [Character]) -> Bool {
|
||||
guard keys.allSatisfy({ $0.isASCII && $0.isLetter }) else { return false }
|
||||
let p = ZhuyinParser.parse(keys + [" "])
|
||||
guard p.pending == nil, p.syllables.count == 1 else { return false }
|
||||
guard let best = classifier.lexicon.entries(for: p.syllables.map(\.syllable)).first else { return false }
|
||||
return best.score >= CompositionEngine.holdMinScore
|
||||
}
|
||||
|
||||
/// Current split of the buffer into Zhuyin / literal runs.
|
||||
public var segments: [Segment] { segments(of: keys) }
|
||||
|
||||
@@ -84,6 +119,29 @@ public final class CompositionEngine {
|
||||
public var markedText: String { markedPieces.map(\.text).joined() }
|
||||
|
||||
public var markedPieces: [MarkedPiece] {
|
||||
var pieces = segmentPieces
|
||||
if !inserting.isEmpty {
|
||||
// Show the syllable being inserted at the cursor.
|
||||
var at = baseCursorOffset, out: [MarkedPiece] = [], done = false
|
||||
for piece in pieces {
|
||||
if !done && at <= piece.text.count {
|
||||
let i = piece.text.index(piece.text.startIndex, offsetBy: at)
|
||||
out.append(MarkedPiece(text: String(piece.text[..<i]), mode: piece.mode))
|
||||
out.append(MarkedPiece(text: insertingText, mode: .zhuyin))
|
||||
out.append(MarkedPiece(text: String(piece.text[i...]), mode: piece.mode))
|
||||
done = true
|
||||
} else {
|
||||
out.append(piece)
|
||||
at -= piece.text.count
|
||||
}
|
||||
}
|
||||
pieces = out.filter { !$0.text.isEmpty }
|
||||
}
|
||||
if pendingBacktick { pieces.append(MarkedPiece(text: "`", mode: .literal)) }
|
||||
return pieces
|
||||
}
|
||||
|
||||
private var segmentPieces: [MarkedPiece] {
|
||||
let segs = segments
|
||||
return segs.enumerated().map { i, seg in
|
||||
MarkedPiece(text: render(seg, final: i < segs.count - 1), mode: seg.mode)
|
||||
@@ -113,87 +171,202 @@ public final class CompositionEngine {
|
||||
|
||||
// MARK: - Cursor
|
||||
|
||||
/// A place the cursor can stop: after the `local`-th syllable of Zhuyin segment `segIndex`.
|
||||
/// Absolute key index after each syllable of a Zhuyin segment.
|
||||
private func syllableEnds(_ seg: Segment) -> [Int] {
|
||||
ZhuyinParser.parse(Array(keys[seg.range])).syllables.map { seg.range.lowerBound + $0.keyRange.upperBound }
|
||||
}
|
||||
|
||||
/// Key indices the cursor can stop at: the start, after every Zhuyin syllable, and at
|
||||
/// every segment boundary.
|
||||
private var cursorPositions: [Int] {
|
||||
var out: Set<Int> = [0, keys.count]
|
||||
for seg in segments {
|
||||
out.insert(seg.range.upperBound)
|
||||
if seg.mode == .zhuyin { out.formUnion(syllableEnds(seg)) }
|
||||
}
|
||||
return out.sorted()
|
||||
}
|
||||
|
||||
/// A place to open candidates: after the `local`-th syllable of Zhuyin segment `segIndex`.
|
||||
private struct CursorStop { let segIndex: Int; let local: Int }
|
||||
|
||||
private var cursorStops: [CursorStop] {
|
||||
private func stop(at pos: Int) -> CursorStop? {
|
||||
let segs = segments
|
||||
var out: [CursorStop] = []
|
||||
for (i, seg) in segs.enumerated() where seg.mode == .zhuyin {
|
||||
for (i, seg) in segs.enumerated() where seg.mode == .zhuyin
|
||||
&& seg.range.lowerBound < pos && pos <= seg.range.upperBound {
|
||||
let n = convertibleSyllables(seg, final: i < segs.count - 1).syllables.count
|
||||
for k in stride(from: 1, through: n, by: 1) { out.append(CursorStop(segIndex: i, local: k)) }
|
||||
let local = syllableEnds(seg).filter { $0 <= pos }.count
|
||||
return local > 0 ? CursorStop(segIndex: i, local: min(local, n)) : nil
|
||||
}
|
||||
return out
|
||||
return nil
|
||||
}
|
||||
|
||||
/// Where ↓ opens candidates: before the cursor, or the last syllable.
|
||||
private var candidateStop: CursorStop? {
|
||||
if let c = cursor { return stop(at: c) }
|
||||
let segs = segments
|
||||
for i in segs.indices.reversed() where segs[i].mode == .zhuyin {
|
||||
let n = convertibleSyllables(segs[i], final: i < segs.count - 1).syllables.count
|
||||
if n > 0 { return CursorStop(segIndex: i, local: n) }
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
/// Cursor position in the rendered segments (without the syllable being inserted).
|
||||
private var baseCursorOffset: Int {
|
||||
let pieces = segmentPieces
|
||||
guard let c = cursor else { return pieces.reduce(0) { $0 + $1.text.count } }
|
||||
let segs = segments
|
||||
for (i, seg) in segs.enumerated() where seg.range.lowerBound < c && c <= seg.range.upperBound {
|
||||
var offset = pieces[..<i].reduce(0) { $0 + $1.text.count }
|
||||
if seg.mode == .literal { return offset + c - seg.range.lowerBound }
|
||||
let local = stop(at: c)?.local ?? 0
|
||||
for conv in conversion(of: seg, final: i < segs.count - 1).segments {
|
||||
if conv.range.upperBound <= local {
|
||||
offset += conv.text.count
|
||||
} else if conv.range.lowerBound < local {
|
||||
offset += min(local - conv.range.lowerBound, conv.text.count)
|
||||
}
|
||||
}
|
||||
return offset
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
/// Cursor position in `markedText`, in Characters.
|
||||
public var cursorOffset: Int {
|
||||
let pieces = markedPieces
|
||||
let total = pieces.reduce(0) { $0 + $1.text.count }
|
||||
let stops = cursorStops
|
||||
guard let c = cursor, stops.indices.contains(c) else { return total }
|
||||
let stop = stops[c]
|
||||
var offset = pieces[..<stop.segIndex].reduce(0) { $0 + $1.text.count }
|
||||
let segs = segments
|
||||
for conv in conversion(of: segs[stop.segIndex], final: stop.segIndex < segs.count - 1).segments {
|
||||
if conv.range.upperBound <= stop.local {
|
||||
offset += conv.text.count
|
||||
} else if conv.range.lowerBound < stop.local {
|
||||
offset += min(stop.local - conv.range.lowerBound, conv.text.count)
|
||||
}
|
||||
}
|
||||
return offset
|
||||
public var cursorOffset: Int { baseCursorOffset + insertingText.count }
|
||||
|
||||
private var insertingText: String {
|
||||
inserting.isEmpty ? "" : ZhuyinParser.parse(inserting).syllables.map(\.syllable.reading).joined()
|
||||
}
|
||||
|
||||
private func moveCursor(by delta: Int) {
|
||||
let stops = cursorStops
|
||||
guard !stops.isEmpty else { return }
|
||||
let next = (cursor ?? stops.count - 1) + delta
|
||||
cursor = next >= stops.count - 1 ? nil : max(0, next)
|
||||
inserting = []
|
||||
let stops = cursorPositions
|
||||
let here = cursor ?? keys.count
|
||||
let next = delta < 0 ? (stops.last { $0 < here } ?? here) : (stops.first { $0 > here } ?? here)
|
||||
cursor = next >= keys.count ? nil : next
|
||||
}
|
||||
|
||||
// MARK: - Editing in the middle
|
||||
|
||||
/// Keeps picked phrases that end at or before key index `pos` (they don't move).
|
||||
private func trimFixed(at pos: Int) {
|
||||
var kept: [Int: [Range<Int>: String]] = [:]
|
||||
for seg in segments where seg.mode == .zhuyin {
|
||||
guard let picks = fixed[seg.range.lowerBound] else { continue }
|
||||
if seg.range.upperBound <= pos {
|
||||
kept[seg.range.lowerBound] = picks
|
||||
} else if seg.range.lowerBound < pos {
|
||||
let ends = syllableEnds(seg)
|
||||
let here = picks.filter { $0.key.upperBound <= ends.count && ends[$0.key.upperBound - 1] <= pos }
|
||||
if !here.isEmpty { kept[seg.range.lowerBound] = here }
|
||||
}
|
||||
}
|
||||
fixed = kept
|
||||
}
|
||||
|
||||
/// Inserts keys at the cursor and moves the cursor after them.
|
||||
private func splice(_ newKeys: [Character]) {
|
||||
guard let pos = cursor else { keys += newKeys; return }
|
||||
trimFixed(at: pos)
|
||||
keys.insert(contentsOf: newKeys, at: pos)
|
||||
if let h = held, h >= pos { held = h + newKeys.count }
|
||||
let next = pos + newKeys.count
|
||||
cursor = next >= keys.count ? nil : next
|
||||
}
|
||||
|
||||
/// Backspace in the middle: removes the syllable (or literal key) before the cursor.
|
||||
private func deleteBeforeCursor(_ pos: Int) {
|
||||
var start = pos - 1
|
||||
for seg in segments where seg.mode == .zhuyin && seg.range.lowerBound < pos && pos <= seg.range.upperBound {
|
||||
for ps in ZhuyinParser.parse(Array(keys[seg.range])).syllables
|
||||
where seg.range.lowerBound + ps.keyRange.upperBound == pos {
|
||||
start = seg.range.lowerBound + ps.keyRange.lowerBound
|
||||
}
|
||||
}
|
||||
trimFixed(at: start)
|
||||
keys.removeSubrange(start..<pos)
|
||||
if let h = held {
|
||||
if h >= start && h < pos { held = nil } else if h >= pos { held = h - (pos - start) }
|
||||
}
|
||||
cursor = start
|
||||
if keys.isEmpty { reset() }
|
||||
}
|
||||
|
||||
/// Typing with the cursor in the middle: compose one syllable, splice it in on its tone.
|
||||
private func insertKey(_ c: Character) -> KeyResult {
|
||||
if c == " " {
|
||||
if inserting.isEmpty { return KeyResult(handled: true, commit: commitCurrent()) }
|
||||
splice(inserting + [" "])
|
||||
inserting = []
|
||||
return .consumed
|
||||
}
|
||||
inserting.append(c)
|
||||
let p = ZhuyinParser.parse(inserting)
|
||||
if p.strayTones > 0 || p.foreignKeys > 0 {
|
||||
inserting.removeLast() // a tone with nothing to attach to
|
||||
} else if p.syllables.count == 1, p.syllables[0].toned {
|
||||
splice(inserting)
|
||||
inserting = []
|
||||
} else if p.syllables.count > 1 {
|
||||
// A new syllable started without a tone: splice the first one as it is.
|
||||
let first = p.syllables[0].keyRange.upperBound
|
||||
splice(Array(inserting[..<first]))
|
||||
inserting = Array(inserting[first...])
|
||||
}
|
||||
return .consumed
|
||||
}
|
||||
|
||||
// MARK: - Key handling
|
||||
|
||||
public func handle(_ key: KeyInput) -> KeyResult {
|
||||
if isShowingCandidates { return handleCandidateKey(key) }
|
||||
if pendingBacktick { return afterBacktick(key) }
|
||||
if !inserting.isEmpty && (key == .backspace || key == .escape) {
|
||||
if key == .backspace { inserting.removeLast() } else { inserting = [] }
|
||||
return .consumed
|
||||
}
|
||||
|
||||
switch key {
|
||||
case .character(let c):
|
||||
return handleCharacter(c)
|
||||
case .backspace:
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
cursor = nil
|
||||
keys.removeLast()
|
||||
fixed = fixed.filter { $0.key < keys.count }
|
||||
if keys.isEmpty {
|
||||
// A Shift-tap "next word is English" stays armed; a correction is dropped.
|
||||
if forcedIsCorrection { forcedMode = nil; forcedIsCorrection = false }
|
||||
fixed = [:]
|
||||
if let c = cursor {
|
||||
if c > 0 { deleteBeforeCursor(c) }
|
||||
return .consumed
|
||||
}
|
||||
keys.removeLast()
|
||||
if let h = held, keys.count <= h { held = nil }
|
||||
fixed = fixed.filter { $0.key < keys.count }
|
||||
if keys.isEmpty { forcedMode = nil; fixed = [:] }
|
||||
return .consumed
|
||||
case .enter:
|
||||
guard !keys.isEmpty else { endDirectEnglish(); return .passThrough }
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
if !inserting.isEmpty { splice(inserting); inserting = [] }
|
||||
return KeyResult(handled: true, commit: commitCurrent())
|
||||
case .escape:
|
||||
guard !keys.isEmpty else { endDirectEnglish(); return .passThrough }
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
reset()
|
||||
return .consumed
|
||||
case .tab:
|
||||
guard !keys.isEmpty else { endDirectEnglish(); return .passThrough }
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
flip()
|
||||
return .consumed
|
||||
case .shiftTap:
|
||||
if keys.isEmpty {
|
||||
// Next word is English, typed as-is (lowercase stays lowercase). Tap again to cancel.
|
||||
forcedMode = forcedMode == .literal ? nil : .literal
|
||||
forcedIsCorrection = false
|
||||
// Toggle English mode; it stays on until the next Shift tap.
|
||||
isEnglishMode.toggle()
|
||||
previousMode = isEnglishMode ? .literal : nil
|
||||
previousCommitTime = now()
|
||||
} else {
|
||||
flip()
|
||||
}
|
||||
return .consumed
|
||||
case .down:
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
openCandidates()
|
||||
if inserting.isEmpty { openCandidates() }
|
||||
return .consumed
|
||||
case .left:
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
@@ -211,41 +384,29 @@ public final class CompositionEngine {
|
||||
/// Forces the whole buffer into the other mode (a correction that is learned).
|
||||
private func flip() {
|
||||
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
|
||||
forcedIsCorrection = true
|
||||
}
|
||||
|
||||
/// True while the user has forced English for the next word (Shift tap). In this
|
||||
/// mode keys go straight to the app with no composition (no underline).
|
||||
public var isEnglishArmed: Bool { keys.isEmpty && forcedMode == .literal && !forcedIsCorrection }
|
||||
|
||||
/// Leaves forced-English mode once the word is done.
|
||||
private func endDirectEnglish() {
|
||||
guard isEnglishArmed else { return }
|
||||
forcedMode = nil
|
||||
previousMode = .literal
|
||||
previousCommitTime = now()
|
||||
}
|
||||
|
||||
private static let englishWordPunctuation: Set<Character> = ["'", ".", ",", "-", "/", ";"]
|
||||
|
||||
private func handleCharacter(_ c: Character) -> KeyResult {
|
||||
cursor = nil // typing always continues at the end
|
||||
if isEnglishArmed {
|
||||
// Forced English: insert directly. Space ends the word; other symbols end it too.
|
||||
if c == " " {
|
||||
endDirectEnglish()
|
||||
return KeyResult(handled: true, commit: " ")
|
||||
}
|
||||
if c.isASCII && (c.isLetter || c.isNumber || CompositionEngine.englishWordPunctuation.contains(c)) {
|
||||
return KeyResult(handled: true, commit: String(c))
|
||||
}
|
||||
endDirectEnglish()
|
||||
return .passThrough
|
||||
if cursor != nil, !keys.isEmpty, c == " " || DachenLayout.isZhuyinKey(c) {
|
||||
return insertKey(c)
|
||||
}
|
||||
if !inserting.isEmpty { splice(inserting); inserting = [] }
|
||||
cursor = nil // anything else continues at the end
|
||||
// English mode: every key goes to the app as typed.
|
||||
if isEnglishMode && keys.isEmpty { return .passThrough }
|
||||
if c == "`" {
|
||||
pendingBacktick = true
|
||||
return .consumed
|
||||
}
|
||||
if c == " " {
|
||||
guard !keys.isEmpty else { return .passThrough }
|
||||
let segs = segments(of: keys, finishing: true)
|
||||
if segs.last?.mode == .literal {
|
||||
if held == nil, segs.count == 1, forcedMode == nil, isAmbiguousWord(keys) {
|
||||
held = keys.count
|
||||
keys.append(" ")
|
||||
return .consumed
|
||||
}
|
||||
return KeyResult(handled: true, commit: commitCurrent(segs) + " ")
|
||||
}
|
||||
if parse(ofLast: segs)?.pending != nil {
|
||||
@@ -282,6 +443,26 @@ public final class CompositionEngine {
|
||||
return ZhuyinParser.parse(Array(keys[last.range]))
|
||||
}
|
||||
|
||||
private func afterBacktick(_ key: KeyInput) -> KeyResult {
|
||||
pendingBacktick = false
|
||||
if key == .escape || key == .backspace { return .consumed }
|
||||
let committed = keys.isEmpty ? "" : commitCurrent()
|
||||
if case .character(let c) = key, let symbol = CompositionEngine.symbols[c] {
|
||||
return KeyResult(handled: true, commit: committed + symbol)
|
||||
}
|
||||
// Not a symbol: the backtick was meant literally; then handle the key as usual.
|
||||
var r = handle(key)
|
||||
if !r.handled, case .character(let c) = key { r = KeyResult(handled: true, commit: String(c)) }
|
||||
return KeyResult(handled: r.handled, commit: committed + "`" + (r.commit ?? ""))
|
||||
}
|
||||
|
||||
/// "`" + key → full-width symbol.
|
||||
static let symbols: [Character: String] = [
|
||||
",": ",", "<": ",", ".": "。", ">": "。", "/": "?", "?": "?", ";": ";", ":": ":",
|
||||
"'": "、", "\\": "、", "\"": "「", "[": "「", "]": "」", "{": "『", "}": "』",
|
||||
"(": "(", ")": ")", "!": "!", "~": "~", "-": "——", "_": "——", "=": "=",
|
||||
]
|
||||
|
||||
static let fullwidth: [Character: String] = [
|
||||
"<": ",", ">": "。", "?": "?", "!": "!", ":": ":", "\"": "「", "(": "(", ")": ")",
|
||||
"{": "『", "}": "』", "[": "「", "]": "」", "~": "~", "_": "——",
|
||||
@@ -291,9 +472,7 @@ public final class CompositionEngine {
|
||||
|
||||
/// Opens candidates for the phrase ending at the cursor (default: the last syllable).
|
||||
private func openCandidates() {
|
||||
let stops = cursorStops
|
||||
guard !stops.isEmpty else { return }
|
||||
let stop = stops[min(cursor ?? stops.count - 1, stops.count - 1)]
|
||||
guard let stop = candidateStop else { return }
|
||||
let segs = segments
|
||||
let seg = segs[stop.segIndex]
|
||||
let syl = convertibleSyllables(seg, final: stop.segIndex < segs.count - 1).syllables
|
||||
@@ -338,7 +517,7 @@ public final class CompositionEngine {
|
||||
private func commitCurrent(_ given: [Segment]? = nil) -> String {
|
||||
let segs = given ?? segments
|
||||
let text = segs.map { render($0, final: true) }.joined()
|
||||
if let forced = forcedMode, forcedIsCorrection {
|
||||
if let forced = forcedMode {
|
||||
classifier.learned.record(String(keys), as: forced)
|
||||
}
|
||||
previousMode = segs.last?.mode
|
||||
@@ -354,9 +533,11 @@ public final class CompositionEngine {
|
||||
|
||||
public func reset() {
|
||||
keys = []
|
||||
inserting = []
|
||||
held = nil
|
||||
pendingBacktick = false
|
||||
cursor = nil
|
||||
forcedMode = nil
|
||||
forcedIsCorrection = false
|
||||
fixed = [:]
|
||||
candidates = []
|
||||
}
|
||||
|
||||
@@ -8,33 +8,63 @@ import Foundation
|
||||
/// - a letter-bigram model trained on those words, for words not in either list.
|
||||
public final class EnglishModel {
|
||||
private var common = Set<String>()
|
||||
private var dictionary = Set<String>()
|
||||
private var sortedWords: [String] = []
|
||||
private var sortedCommon: [String] = []
|
||||
/// Dictionary words (lowercase, > 2 letters), sorted: membership and prefix checks are
|
||||
/// binary searches, so no 230k-entry Set has to be built at startup.
|
||||
private var sortedDict: [String] = []
|
||||
private var bigram: [String: Double] = [:] // "ab" -> log2 P(b|a); "^a", "a$" for boundaries
|
||||
private var unseenBigram = -12.0
|
||||
|
||||
public init(commonWords: [String], dictionaryWords: [String] = []) {
|
||||
/// - Parameter bigram: precomputed table (english-bigram.json); trained from the words if nil.
|
||||
public init(commonWords: [String], dictionaryWords: [String] = [],
|
||||
bigram: (table: [String: Double], unseen: Double)? = nil) {
|
||||
common = Set(commonWords.map { $0.lowercased() }.filter { !$0.isEmpty })
|
||||
dictionary = Set(dictionaryWords.map { $0.lowercased() }.filter { $0.count > 2 })
|
||||
sortedCommon = common.sorted()
|
||||
sortedWords = common.union(dictionary).sorted()
|
||||
trainBigram()
|
||||
var dict = dictionaryWords.map { $0.lowercased() }.filter { $0.count > 2 }
|
||||
if !zip(dict, dict.dropFirst()).allSatisfy({ $0 < $1 }) {
|
||||
dict = Array(Set(dict)).sorted()
|
||||
}
|
||||
sortedDict = dict
|
||||
if let bigram {
|
||||
self.bigram = bigram.table
|
||||
unseenBigram = bigram.unseen
|
||||
} else {
|
||||
trainBigram()
|
||||
}
|
||||
}
|
||||
|
||||
/// Loads common words from a bundle resource file and the system dictionary if present.
|
||||
public convenience init(commonWordsURL: URL?, systemDictionary: URL? = URL(fileURLWithPath: "/usr/share/dict/words")) {
|
||||
/// Loads the bundled data files. Falls back to the system word list and to training
|
||||
/// the bigram table when the precomputed files are missing.
|
||||
public convenience init(commonWordsURL: URL?, dictionaryURL: URL? = nil, bigramURL: URL? = nil) {
|
||||
func lines(_ url: URL?) -> [String] {
|
||||
guard let url, let text = try? String(contentsOf: url, encoding: .utf8) else { return [] }
|
||||
return text.split(whereSeparator: \.isNewline)
|
||||
.map { $0.trimmingCharacters(in: .whitespaces) }
|
||||
.filter { !$0.hasPrefix("#") }
|
||||
.filter { !$0.isEmpty && !$0.hasPrefix("#") }
|
||||
}
|
||||
self.init(commonWords: lines(commonWordsURL), dictionaryWords: lines(systemDictionary))
|
||||
var bigram: (table: [String: Double], unseen: Double)?
|
||||
if let url = bigramURL, let data = try? Data(contentsOf: url),
|
||||
let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
|
||||
let table = obj["table"] as? [String: Double], let unseen = obj["unseen"] as? Double {
|
||||
bigram = (table, unseen)
|
||||
}
|
||||
let dictURL = dictionaryURL.flatMap { FileManager.default.fileExists(atPath: $0.path) ? $0 : nil }
|
||||
?? URL(fileURLWithPath: "/usr/share/dict/words")
|
||||
self.init(commonWords: lines(commonWordsURL), dictionaryWords: lines(dictURL), bigram: bigram)
|
||||
}
|
||||
|
||||
public func isCommon(_ w: String) -> Bool { common.contains(w.lowercased()) }
|
||||
public func isWord(_ w: String) -> Bool { let l = w.lowercased(); return common.contains(l) || dictionary.contains(l) }
|
||||
public func isWord(_ w: String) -> Bool { let l = w.lowercased(); return common.contains(l) || inDictionary(l) }
|
||||
|
||||
private func inDictionary(_ w: String) -> Bool {
|
||||
let i = lowerBound(w, in: sortedDict)
|
||||
return i < sortedDict.count && sortedDict[i] == w
|
||||
}
|
||||
|
||||
/// Some word strictly longer than `p` starts with it (common words or dictionary).
|
||||
private func isPrefixOfWord(_ p: String) -> Bool {
|
||||
hasPrefix(p, in: sortedCommon) || hasPrefix(p, in: sortedDict)
|
||||
}
|
||||
|
||||
/// - Parameter complete: true when the user ended the word (space / punctuation),
|
||||
/// false while still typing (prefixes of words get partial credit).
|
||||
@@ -43,10 +73,10 @@ public final class EnglishModel {
|
||||
let n = Double(w.count)
|
||||
guard !w.isEmpty else { return 0 }
|
||||
if common.contains(w) { return 3 + n }
|
||||
if dictionary.contains(w) { return 1 + 0.6 * n }
|
||||
if inDictionary(w) { return 1 + 0.6 * n }
|
||||
if !complete {
|
||||
if hasPrefix(w, in: sortedCommon) { return 0.4 * n }
|
||||
if hasPrefix(w, in: sortedWords) { return 0.2 * n }
|
||||
if isPrefixOfWord(w) { return 0.2 * n }
|
||||
}
|
||||
// Unknown word: judge by how English-like its letter sequence is.
|
||||
let plaus = plausibility(w)
|
||||
@@ -63,12 +93,12 @@ public final class EnglishModel {
|
||||
if common.contains(w) {
|
||||
f["lit.common"] = 1
|
||||
f["lit.common_len"] = n
|
||||
} else if dictionary.contains(w) {
|
||||
} else if inDictionary(w) {
|
||||
f["lit.dict"] = 1
|
||||
f["lit.dict_len"] = n
|
||||
} else if !complete && hasPrefix(w, in: sortedCommon) {
|
||||
f["lit.prefix_common_len"] = n
|
||||
} else if !complete && hasPrefix(w, in: sortedWords) {
|
||||
} else if !complete && isPrefixOfWord(w) {
|
||||
f["lit.prefix_dict_len"] = n
|
||||
} else {
|
||||
let plaus = plausibility(w)
|
||||
@@ -85,12 +115,17 @@ public final class EnglishModel {
|
||||
return total / Double(chars.count - 1)
|
||||
}
|
||||
|
||||
private func hasPrefix(_ p: String, in words: [String]) -> Bool {
|
||||
private func lowerBound(_ p: String, in words: [String]) -> Int {
|
||||
var lo = 0, hi = words.count
|
||||
while lo < hi {
|
||||
let mid = (lo + hi) / 2
|
||||
if words[mid] < p { lo = mid + 1 } else { hi = mid }
|
||||
}
|
||||
return lo
|
||||
}
|
||||
|
||||
private func hasPrefix(_ p: String, in words: [String]) -> Bool {
|
||||
var lo = lowerBound(p, in: words)
|
||||
// words[lo] is the first word >= p; skip an exact match, then test the next one.
|
||||
if lo < words.count && words[lo] == p { lo += 1 }
|
||||
return lo < words.count && words[lo].hasPrefix(p)
|
||||
@@ -99,7 +134,7 @@ public final class EnglishModel {
|
||||
private func trainBigram() {
|
||||
var pair: [String: Double] = [:]
|
||||
var first: [String: Double] = [:]
|
||||
for w in sortedWords where w.allSatisfy({ $0.isLetter && $0.isASCII }) {
|
||||
for w in Set(sortedCommon + sortedDict).sorted() where w.allSatisfy({ $0.isLetter && $0.isASCII }) {
|
||||
let weight = common.contains(w) ? 5.0 : 1.0
|
||||
let chars = ["^"] + w.map(String.init) + ["$"]
|
||||
for i in 1..<chars.count {
|
||||
|
||||
@@ -42,6 +42,9 @@ public enum Features {
|
||||
"lit.others": -1.5,
|
||||
// Segmentation
|
||||
"seg.switch": -3.0,
|
||||
// Context: the first segment has the same mode as the previously committed text
|
||||
"ctx.zh": 1.5,
|
||||
"ctx.lit": 1.5,
|
||||
]
|
||||
|
||||
/// Reads {"weights": {...}} from a JSON file; empty if missing or invalid.
|
||||
@@ -58,7 +61,8 @@ public enum Features {
|
||||
|
||||
/// Features of `keys` read as Zhuyin, or nil if impossible.
|
||||
public static func zhuyin(_ keys: [Character], lexicon: Lexicon, isLast: Bool = true,
|
||||
afterOther: Bool = false, finishing: Bool = false) -> FeatureVector? {
|
||||
afterOther: Bool = false, finishing: Bool = false,
|
||||
previous: InputMode? = nil) -> FeatureVector? {
|
||||
let parse = ZhuyinParser.parse(keys)
|
||||
// A Zhuyin segment followed by another must end with a tone.
|
||||
if !isLast && parse.pending != nil { return nil }
|
||||
@@ -107,18 +111,27 @@ public enum Features {
|
||||
if isLast && afterOther && !finishing && !parse.syllables.contains(where: \.toned) {
|
||||
add("zh.unconfirmed")
|
||||
}
|
||||
if afterOther { add("seg.switch") }
|
||||
if afterOther {
|
||||
add("seg.switch")
|
||||
} else if previous == .zhuyin {
|
||||
add("ctx.zh")
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
/// Features of `keys` output as typed, or nil if impossible.
|
||||
public static func literal(_ keys: [Character], english: EnglishModel, isLast: Bool = true,
|
||||
afterOther: Bool = false, finishing: Bool = false) -> FeatureVector? {
|
||||
afterOther: Bool = false, finishing: Bool = false,
|
||||
previous: InputMode? = nil) -> FeatureVector? {
|
||||
if keys.isEmpty || keys.contains(" ") { return nil }
|
||||
let complete = !isLast || finishing
|
||||
var f: FeatureVector = ["lit.bias": 1]
|
||||
func add(_ k: String, _ v: Double = 1) { f[k, default: 0] += v }
|
||||
if afterOther { add("seg.switch") }
|
||||
if afterOther {
|
||||
add("seg.switch")
|
||||
} else if previous == .literal {
|
||||
add("ctx.lit")
|
||||
}
|
||||
if keys.contains(where: { $0.isUppercase }) { add("lit.upper") }
|
||||
|
||||
let letters = String(keys.filter { $0.isLetter })
|
||||
|
||||
@@ -11,58 +11,159 @@ public struct LexiconEntry: Equatable {
|
||||
/// File format, one entry per line (`#` starts a comment):
|
||||
/// ㄋㄧˇ-ㄏㄠˇ 你好 -4.2
|
||||
/// The phrase/reading columns may be in either order; the score is optional.
|
||||
///
|
||||
/// Loading is on the input method's startup path, so the file is parsed as raw bytes,
|
||||
/// and the tone-ignoring index (only needed for untoned syllables) is built lazily —
|
||||
/// call `prepareTonelessIndex()` from a background queue to have it ready.
|
||||
public final class Lexicon {
|
||||
private var exact: [String: [LexiconEntry]] = [:]
|
||||
private var toneless: [String: [LexiconEntry]] = [:]
|
||||
private var tonelessIndex: [String: [LexiconEntry]]?
|
||||
private let lock = NSLock()
|
||||
public private(set) var maxPhraseLength = 1
|
||||
|
||||
public init() {}
|
||||
|
||||
public convenience init(contentsOf url: URL) throws {
|
||||
self.init()
|
||||
load(try String(contentsOf: url, encoding: .utf8))
|
||||
load(data: try Data(contentsOf: url))
|
||||
}
|
||||
|
||||
public func load(_ text: String) {
|
||||
for line in text.split(whereSeparator: \.isNewline) {
|
||||
let trimmed = line.trimmingCharacters(in: .whitespaces)
|
||||
if trimmed.isEmpty || trimmed.hasPrefix("#") { continue }
|
||||
let cols = trimmed.split(whereSeparator: { $0 == " " || $0 == "\t" }).map(String.init)
|
||||
guard cols.count >= 2 else { continue }
|
||||
let (reading, phrase) = Lexicon.isBopomofo(cols[0]) ? (cols[0], cols[1]) : (cols[1], cols[0])
|
||||
guard Lexicon.isBopomofo(reading) else { continue }
|
||||
let score = cols.count >= 3 ? Double(cols[2]) ?? -6 : -6
|
||||
add(reading: reading, phrase: phrase, score: score)
|
||||
}
|
||||
for key in exact.keys { exact[key]?.sort { $0.score > $1.score } }
|
||||
for key in toneless.keys { toneless[key]?.sort { $0.score > $1.score } }
|
||||
load(data: Data(text.utf8))
|
||||
}
|
||||
|
||||
public func add(reading rawReading: String, phrase: String, score: Double) {
|
||||
let entry = LexiconEntry(phrase: phrase, score: score)
|
||||
public func load(data: Data) {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
exact.reserveCapacity(exact.count + data.count / 30)
|
||||
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
|
||||
let bytes = raw.bindMemory(to: UInt8.self)
|
||||
let n = bytes.count
|
||||
var i = 0
|
||||
while i < n {
|
||||
var end = i
|
||||
while end < n && bytes[end] != 0x0A { end += 1 }
|
||||
parseLine(bytes, i, end)
|
||||
i = end + 1
|
||||
}
|
||||
}
|
||||
for (key, entries) in exact where entries.count > 1 {
|
||||
exact[key] = entries.sorted { $0.score > $1.score }
|
||||
}
|
||||
tonelessIndex = nil // rebuilt on demand
|
||||
}
|
||||
|
||||
/// Splits one line into up to three space/tab separated fields.
|
||||
private func parseLine(_ b: UnsafeBufferPointer<UInt8>, _ start: Int, _ end: Int) {
|
||||
var fields: [Range<Int>] = []
|
||||
var i = start
|
||||
while i < end && fields.count < 3 {
|
||||
while i < end && (b[i] == 0x20 || b[i] == 0x09 || b[i] == 0x0D) { i += 1 }
|
||||
if i >= end { break }
|
||||
if fields.isEmpty && b[i] == 0x23 { return } // "#" comment
|
||||
let s = i
|
||||
while i < end && b[i] != 0x20 && b[i] != 0x09 && b[i] != 0x0D { i += 1 }
|
||||
fields.append(s..<i)
|
||||
}
|
||||
guard fields.count >= 2 else { return }
|
||||
func str(_ r: Range<Int>) -> String { String(decoding: UnsafeBufferPointer(rebasing: b[r]), as: UTF8.self) }
|
||||
let (r, p) = Lexicon.isBopomofo(b, fields[0]) ? (fields[0], fields[1]) : (fields[1], fields[0])
|
||||
guard Lexicon.isBopomofo(b, r) else { return }
|
||||
let score = fields.count >= 3 ? Lexicon.parseDouble(b, fields[2]) ?? -6 : -6
|
||||
// Neutral tone written as a prefix ("˙ㄉㄜ") needs normalising; detect it on bytes.
|
||||
var prefixNeutral = b[r.lowerBound] == 0xCB
|
||||
var k = r.lowerBound
|
||||
while !prefixNeutral && k + 2 < r.upperBound {
|
||||
if b[k] == 0x2D && b[k + 1] == 0xCB && b[k + 2] == 0x99 { prefixNeutral = true }
|
||||
k += 1
|
||||
}
|
||||
addEntry(reading: str(r), phrase: str(p), score: score, normalise: prefixNeutral)
|
||||
}
|
||||
|
||||
/// Parses "-3.9151"-style decimals without creating a String.
|
||||
private static func parseDouble(_ b: UnsafeBufferPointer<UInt8>, _ r: Range<Int>) -> Double? {
|
||||
var i = r.lowerBound
|
||||
var sign = 1.0
|
||||
if i < r.upperBound && (b[i] == 0x2D || b[i] == 0x2B) { sign = b[i] == 0x2D ? -1 : 1; i += 1 }
|
||||
var value = 0.0, scale = 1.0, digits = 0, fraction = false
|
||||
while i < r.upperBound {
|
||||
let c = b[i]
|
||||
if c >= 0x30 && c <= 0x39 {
|
||||
if fraction { scale /= 10; value += Double(c - 0x30) * scale } else { value = value * 10 + Double(c - 0x30) }
|
||||
digits += 1
|
||||
} else if c == 0x2E && !fraction {
|
||||
fraction = true
|
||||
} else {
|
||||
return Double(String(decoding: UnsafeBufferPointer(rebasing: b[r]), as: UTF8.self))
|
||||
}
|
||||
i += 1
|
||||
}
|
||||
return digits > 0 ? sign * value : nil
|
||||
}
|
||||
|
||||
/// Adds one entry (call `load` for bulk data; this does not re-sort).
|
||||
public func add(reading: String, phrase: String, score: Double) {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
let key = addEntry(reading: reading, phrase: phrase, score: score, normalise: reading.contains("˙"))
|
||||
exact[key]?.sort { $0.score > $1.score }
|
||||
tonelessIndex = nil
|
||||
}
|
||||
|
||||
@discardableResult
|
||||
private func addEntry(reading rawReading: String, phrase: String, score: Double, normalise: Bool) -> String {
|
||||
var reading = rawReading
|
||||
// Some data sets write the neutral tone as a prefix ("˙ㄉㄜ"); we use a suffix.
|
||||
let syllables = rawReading.split(separator: "-").map { s -> String in
|
||||
s.hasPrefix("˙") ? String(s.dropFirst()) + "˙" : String(s)
|
||||
if normalise {
|
||||
reading = reading.split(separator: "-").map { s -> String in
|
||||
s.hasPrefix("˙") ? String(s.dropFirst()) + "˙" : String(s)
|
||||
}.joined(separator: "-")
|
||||
}
|
||||
let reading = syllables.joined(separator: "-")
|
||||
maxPhraseLength = max(maxPhraseLength, syllables.count)
|
||||
if !(exact[reading]?.contains { $0.phrase == phrase } ?? false) {
|
||||
exact[reading, default: []].append(entry)
|
||||
maxPhraseLength = max(maxPhraseLength, reading.utf8.reduce(1) { $0 + ($1 == 0x2D ? 1 : 0) })
|
||||
if exact[reading] == nil {
|
||||
exact[reading] = [LexiconEntry(phrase: phrase, score: score)]
|
||||
} else if !exact[reading]!.contains(where: { $0.phrase == phrase }) {
|
||||
exact[reading]!.append(LexiconEntry(phrase: phrase, score: score))
|
||||
}
|
||||
let key = Lexicon.stripTones(reading)
|
||||
if let i = toneless[key]?.firstIndex(where: { $0.phrase == phrase }) {
|
||||
toneless[key]![i].score = max(toneless[key]![i].score, score)
|
||||
} else {
|
||||
toneless[key, default: []].append(entry)
|
||||
return reading
|
||||
}
|
||||
|
||||
/// Builds the index that ignores tones (for untoned syllables). Safe to call from a
|
||||
/// background queue; lookups build it themselves if it is not ready yet.
|
||||
public func prepareTonelessIndex() {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
_ = buildTonelessIfNeeded()
|
||||
}
|
||||
|
||||
private func buildTonelessIfNeeded() -> [String: [LexiconEntry]] {
|
||||
if let index = tonelessIndex { return index }
|
||||
var best: [String: [String: Double]] = [:]
|
||||
for (reading, entries) in exact {
|
||||
let key = Lexicon.stripTones(reading)
|
||||
for e in entries {
|
||||
if let s = best[key]?[e.phrase], s >= e.score { continue }
|
||||
best[key, default: [:]][e.phrase] = e.score
|
||||
}
|
||||
}
|
||||
var index: [String: [LexiconEntry]] = [:]
|
||||
index.reserveCapacity(best.count)
|
||||
for (key, phrases) in best {
|
||||
index[key] = phrases.map { LexiconEntry(phrase: $0.key, score: $0.value) }
|
||||
.sorted { $0.score != $1.score ? $0.score > $1.score : $0.phrase < $1.phrase }
|
||||
}
|
||||
tonelessIndex = index
|
||||
return index
|
||||
}
|
||||
|
||||
/// Entries for a sequence of readings. Untoned syllables match any tone.
|
||||
public func entries(for syllables: [Syllable]) -> [LexiconEntry] {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
if syllables.allSatisfy({ $0.tone != nil }) {
|
||||
return exact[syllables.map(\.reading).joined(separator: "-")] ?? []
|
||||
}
|
||||
return toneless[syllables.map(\.toneless).joined(separator: "-")] ?? []
|
||||
return buildTonelessIfNeeded()[syllables.map(\.toneless).joined(separator: "-")] ?? []
|
||||
}
|
||||
|
||||
public func bestScore(for syllable: Syllable) -> Double? {
|
||||
@@ -81,4 +182,13 @@ public final class Lexicon {
|
||||
guard let first = s.unicodeScalars.first else { return false }
|
||||
return (0x3105...0x312F).contains(first.value) || first == "˙"
|
||||
}
|
||||
|
||||
/// Same test on raw UTF-8: Bopomofo U+3105–U+312F is E3 84 85…E3 84 AF; "˙" is CB 99.
|
||||
private static func isBopomofo(_ b: UnsafeBufferPointer<UInt8>, _ r: Range<Int>) -> Bool {
|
||||
guard r.count >= 2 else { return false }
|
||||
if b[r.lowerBound] == 0xCB && b[r.lowerBound + 1] == 0x99 { return true }
|
||||
guard r.count >= 3, b[r.lowerBound] == 0xE3, b[r.lowerBound + 1] == 0x84 else { return false }
|
||||
let c = b[r.lowerBound + 2]
|
||||
return c >= 0x85 && c <= 0xAF
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
自動注音 AutoZhuyin — 第三方授權聲明 / Third-party notices
|
||||
==========================================================
|
||||
|
||||
本軟體包含或使用下列第三方資料。
|
||||
This software includes or uses the following third-party material.
|
||||
|
||||
|
||||
1. 小麥注音 McBopomofo — 詞庫資料 (lexicon data)
|
||||
------------------------------------------------
|
||||
https://github.com/openvanilla/McBopomofo
|
||||
|
||||
lexicon.txt is converted from McBopomofo's data files (BPMFBase.txt,
|
||||
BPMFMappings.txt, phrase.occ) by scripts/convert-mcbopomofo.py. No McBopomofo
|
||||
program code is included.
|
||||
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2011-2026 Mengjuei Hsieh et al.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
|
||||
2. Webster's Second International Dictionary — 英文字典 (English word list)
|
||||
--------------------------------------------------------------------------
|
||||
english-dict.txt is filtered from /usr/share/dict/words (Webster's Second
|
||||
International Dictionary, 1934), which is in the public domain.
|
||||
|
||||
|
||||
3. PIME — Windows 輸入法框架 (input method framework)
|
||||
----------------------------------------------------
|
||||
https://github.com/EasyIME/PIME — GNU LGPL 2.1
|
||||
|
||||
The Windows version runs as a module inside PIME, which users install
|
||||
separately. No PIME code is included in this software.
|
||||
@@ -6,7 +6,10 @@ private let resources = URL(fileURLWithPath: #filePath)
|
||||
.appendingPathComponent("Resources")
|
||||
|
||||
private let sharedLexicon = try! Lexicon(contentsOf: resources.appendingPathComponent("lexicon.txt"))
|
||||
private let sharedEnglish = EnglishModel(commonWordsURL: resources.appendingPathComponent("english-common.txt"))
|
||||
private let windowsData = resources.deletingLastPathComponent().appendingPathComponent("windows/autozhuyin/data")
|
||||
private let sharedEnglish = EnglishModel(commonWordsURL: resources.appendingPathComponent("english-common.txt"),
|
||||
dictionaryURL: windowsData.appendingPathComponent("english-dict.txt"),
|
||||
bigramURL: windowsData.appendingPathComponent("english-bigram.json"))
|
||||
private let sharedWeights = Features.loadWeights(from: resources.appendingPathComponent("classifier-weights.json"))
|
||||
|
||||
final class ParserTests: XCTestCase {
|
||||
@@ -265,10 +268,13 @@ final class ParityTests: XCTestCase {
|
||||
for fx in fixtures {
|
||||
let keys = Array(fx["keys"] as! String)
|
||||
let expected = (fx["segments"] as! [[Any]]).map { "\($0[0])-\($0[1]):\($0[2])" }
|
||||
let got = c.segment(keys, finishing: fx["finishing"] as! Bool).map {
|
||||
let previous = (fx["previous"] as? String).flatMap(InputMode.init(rawValue:))
|
||||
let got = c.segment(keys, finishing: fx["finishing"] as! Bool, previous: previous).map {
|
||||
"\($0.range.lowerBound)-\($0.range.upperBound):\($0.mode.rawValue)"
|
||||
}
|
||||
if got != expected { mismatches.append("\(String(keys)) finishing=\(fx["finishing"]!): \(got) vs \(expected)") }
|
||||
if got != expected {
|
||||
mismatches.append("\(String(keys)) finishing=\(fx["finishing"]!) previous=\(String(describing: previous)): \(got) vs \(expected)")
|
||||
}
|
||||
}
|
||||
XCTAssertGreaterThan(fixtures.count, 300)
|
||||
XCTAssertTrue(mismatches.isEmpty, "\(mismatches.count) mismatches:\n" + mismatches.prefix(10).joined(separator: "\n"))
|
||||
@@ -297,13 +303,16 @@ final class ShiftTapTests: XCTestCase {
|
||||
return out
|
||||
}
|
||||
|
||||
func testShiftTapForcesNextWordEnglishLowercase() {
|
||||
func testShiftTapTogglesEnglishMode() {
|
||||
let e = engine()
|
||||
// "ru" alone would be ㄐㄧ (機); after a Shift tap it stays English, lowercase.
|
||||
XCTAssertEqual(type("⇧ru ", into: e), "ru ")
|
||||
// Automatic detection resumes after the word.
|
||||
XCTAssertFalse(e.isEnglishArmed)
|
||||
XCTAssertEqual(type("su3cl3\n", into: e), "你好")
|
||||
// "ru" alone would be ㄐㄧ (機); in English mode keys pass straight through, lowercase.
|
||||
XCTAssertEqual(type("⇧ru su ", into: e), "ru su ")
|
||||
XCTAssertTrue(e.isEnglishMode) // still English after spaces
|
||||
XCTAssertEqual(e.markedText, "") // never composed: no underline
|
||||
XCTAssertEqual(type("\n", into: e), "\n") // Enter goes to the app, mode stays
|
||||
XCTAssertTrue(e.isEnglishMode)
|
||||
XCTAssertEqual(type("⇧su3cl3\n", into: e), "你好") // tap again: automatic again
|
||||
XCTAssertFalse(e.isEnglishMode)
|
||||
}
|
||||
|
||||
func testShiftTapTwiceCancels() {
|
||||
@@ -316,22 +325,22 @@ final class ShiftTapTests: XCTestCase {
|
||||
XCTAssertEqual(e.mode, .literal)
|
||||
_ = e.handle(.shiftTap)
|
||||
XCTAssertEqual(e.mode, .zhuyin)
|
||||
XCTAssertFalse(e.isEnglishMode) // a flip, not English mode
|
||||
_ = type("\n", into: e)
|
||||
XCTAssertEqual(e.classifier.learned.weights["so"], 1) // a correction is learned
|
||||
}
|
||||
|
||||
func testForcingBeforeTypingIsNotLearned() {
|
||||
func testEnglishModeIsNotLearned() {
|
||||
let e = engine()
|
||||
_ = type("⇧hello ", into: e)
|
||||
XCTAssertNil(e.classifier.learned.weights["hello"])
|
||||
XCTAssertTrue(e.classifier.learned.weights.isEmpty)
|
||||
}
|
||||
|
||||
func testArmingSurvivesBackspace() {
|
||||
func testBackspaceInEnglishModeGoesToApp() {
|
||||
let e = engine()
|
||||
_ = type("⇧r", into: e)
|
||||
_ = e.handle(.backspace)
|
||||
XCTAssertTrue(e.isEnglishArmed)
|
||||
XCTAssertEqual(type("ru ", into: e), "ru ")
|
||||
_ = type("⇧", into: e)
|
||||
XCTAssertFalse(e.handle(.backspace).handled)
|
||||
XCTAssertTrue(e.isEnglishMode)
|
||||
}
|
||||
|
||||
func testDetector() {
|
||||
@@ -360,7 +369,8 @@ final class CursorTests: XCTestCase {
|
||||
func type(_ s: String, into e: CompositionEngine) -> String {
|
||||
var out = ""
|
||||
for c in s {
|
||||
let key: KeyInput = c == "←" ? .left : c == "→" ? .right : c == "↓" ? .down : c == "\n" ? .enter : .character(c)
|
||||
let key: KeyInput = c == "←" ? .left : c == "→" ? .right : c == "↓" ? .down : c == "\n" ? .enter
|
||||
: c == "\u{8}" ? .backspace : .character(c)
|
||||
let r = e.handle(key)
|
||||
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
|
||||
}
|
||||
@@ -374,7 +384,7 @@ final class CursorTests: XCTestCase {
|
||||
_ = type("←", into: e)
|
||||
XCTAssertEqual(e.cursorOffset, 3)
|
||||
_ = type("←←←←", into: e)
|
||||
XCTAssertEqual(e.cursorOffset, 1) // stops after the first character
|
||||
XCTAssertEqual(e.cursorOffset, 0) // stops at the start
|
||||
_ = type("→→→→", into: e)
|
||||
XCTAssertEqual(e.cursorOffset, 4)
|
||||
}
|
||||
@@ -390,12 +400,48 @@ final class CursorTests: XCTestCase {
|
||||
XCTAssertEqual(type("\n", into: e), "今添天氣")
|
||||
}
|
||||
|
||||
func testTypingReturnsCursorToEnd() {
|
||||
func testPunctuationCommitsFromTheMiddle() {
|
||||
let e = engine()
|
||||
_ = type("rup wu0 ←", into: e)
|
||||
XCTAssertEqual(e.cursorOffset, 1)
|
||||
_ = type("wu0 ", into: e)
|
||||
XCTAssertEqual(e.cursorOffset, e.markedText.count)
|
||||
_ = type("?", into: e)
|
||||
XCTAssertEqual(e.markedText, "")
|
||||
}
|
||||
|
||||
func testInsertSyllable() {
|
||||
let e = engine()
|
||||
_ = type("rup fu4←wu", into: e)
|
||||
XCTAssertEqual(e.markedText, "今ㄊㄧ氣")
|
||||
XCTAssertEqual(e.cursorOffset, 3)
|
||||
_ = type("0 ", into: e)
|
||||
XCTAssertEqual(e.markedText, "今天氣")
|
||||
XCTAssertEqual(e.cursorOffset, 2)
|
||||
XCTAssertEqual(type("\n", into: e), "今天氣")
|
||||
}
|
||||
|
||||
func testInsertAtStart() {
|
||||
XCTAssertEqual(type("su3cl3←←ji3\n", into: engine()), "我你好")
|
||||
}
|
||||
|
||||
func testBackspaceDeletesCharacterBeforeCursor() {
|
||||
let e = engine()
|
||||
_ = type("rup wu0 wu0 fu4←←\u{8}", into: e)
|
||||
XCTAssertEqual(e.markedText, "今天氣")
|
||||
XCTAssertEqual(e.cursorOffset, 1)
|
||||
}
|
||||
|
||||
func testBackspaceInsideInsertedSyllable() {
|
||||
let e = engine()
|
||||
_ = type("rup fu4←wu\u{8}", into: e)
|
||||
XCTAssertEqual(e.markedText, "今ㄊ氣")
|
||||
}
|
||||
|
||||
func testKeepsPickedPhraseBeforeCursor() {
|
||||
let e = engine()
|
||||
_ = type("rup wu0 ↓", into: e)
|
||||
e.selectCandidate(e.candidates.firstIndex { $0.text == "添" }!)
|
||||
_ = type("fu4←wu0 ", into: e) // insert 天 between 添 and 氣
|
||||
XCTAssertEqual(String(e.markedText.prefix(3)), "今添天")
|
||||
}
|
||||
|
||||
func testCandidatesForChineseBeforeEnglish() {
|
||||
@@ -406,26 +452,99 @@ final class CursorTests: XCTestCase {
|
||||
}
|
||||
}
|
||||
|
||||
final class DirectEnglishTests: XCTestCase {
|
||||
func testForcedEnglishHasNoComposition() {
|
||||
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||
_ = e.handle(.shiftTap)
|
||||
let r = e.handle(.character("r"))
|
||||
XCTAssertEqual(r.commit, "r") // inserted right away
|
||||
XCTAssertEqual(e.markedText, "") // nothing underlined
|
||||
XCTAssertEqual(e.handle(.character("u")).commit, "u")
|
||||
XCTAssertEqual(e.handle(.character("'")).commit, "'")
|
||||
XCTAssertFalse(e.handle(.backspace).handled) // the app deletes it
|
||||
XCTAssertTrue(e.isEnglishArmed)
|
||||
XCTAssertEqual(e.handle(.character(" ")).commit, " ")
|
||||
XCTAssertFalse(e.isEnglishArmed) // back to automatic
|
||||
}
|
||||
|
||||
func testEnterEndsForcedEnglish() {
|
||||
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||
_ = e.handle(.shiftTap)
|
||||
_ = e.handle(.character("a"))
|
||||
XCTAssertFalse(e.handle(.enter).handled)
|
||||
XCTAssertFalse(e.isEnglishArmed)
|
||||
final class FirstToneAfterEnglishTests: XCTestCase {
|
||||
/// After an English word, a first-tone syllable + space must stay Chinese (機, 吃, 書…).
|
||||
func testFirstToneAfterEnglish() {
|
||||
for keys in ["ru", "t", "gj", "vup", "tj", "el"] {
|
||||
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||
var out = ""
|
||||
for c in "hello " + keys + " " { if let t = e.handle(.character(c)).commit { out += t } }
|
||||
XCTAssertEqual(out, "hello ", keys)
|
||||
XCTAssertEqual(e.markedText.count, 1, keys) // still composing one Chinese character
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
final class HeldWordTests: XCTestCase {
|
||||
func makeEngine() -> CompositionEngine {
|
||||
CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||
}
|
||||
|
||||
func type(_ s: String, into e: CompositionEngine) -> String {
|
||||
var out = ""
|
||||
for c in s {
|
||||
let key: KeyInput = c == "\n" ? .enter : .character(c)
|
||||
let r = e.handle(key)
|
||||
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/// "up " is also ㄧㄣ (因): the next word decides.
|
||||
func testNextWordDecides() {
|
||||
let cases = ["up jo4 ": "因為", "up the ": "up the ", "pick up the ": "pick up the ",
|
||||
"up\n": "up", "i love ": "i love "]
|
||||
for (keys, want) in cases {
|
||||
XCTAssertEqual(type(keys, into: makeEngine()), want, keys)
|
||||
}
|
||||
}
|
||||
|
||||
func testHeldWordStaysComposing() {
|
||||
let e = makeEngine()
|
||||
XCTAssertEqual(type("up ", into: e), "")
|
||||
XCTAssertEqual(e.markedText, "up ")
|
||||
_ = e.handle(.backspace)
|
||||
XCTAssertEqual(e.markedText, "up")
|
||||
XCTAssertEqual(type(" the ", into: e), "up the ")
|
||||
}
|
||||
}
|
||||
|
||||
final class BacktickSymbolTests: XCTestCase {
|
||||
func makeEngine() -> CompositionEngine {
|
||||
CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||
}
|
||||
|
||||
func type(_ s: String, into e: CompositionEngine) -> String {
|
||||
var out = ""
|
||||
for c in s {
|
||||
let key: KeyInput = c == "\n" ? .enter : .character(c)
|
||||
let r = e.handle(key)
|
||||
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func testForcedSymbols() {
|
||||
let cases = ["`<": ",", "`.": "。", "`?": "?", "`;": ";", "`[": "「",
|
||||
"hello`,": "hello,", "su3cl3`.": "你好。", "```\n": "```"]
|
||||
for (keys, want) in cases {
|
||||
XCTAssertEqual(type(keys, into: makeEngine()), want, keys)
|
||||
}
|
||||
}
|
||||
|
||||
func testPlainBacktick() {
|
||||
let e = makeEngine()
|
||||
XCTAssertEqual(type("`", into: e), "")
|
||||
XCTAssertEqual(e.markedText, "`")
|
||||
XCTAssertEqual(type("ls` ", into: e), "`ls` ")
|
||||
}
|
||||
|
||||
func testBackspaceCancels() {
|
||||
let e = makeEngine()
|
||||
_ = type("su3`", into: e)
|
||||
_ = e.handle(.backspace)
|
||||
XCTAssertEqual(e.markedText, "你")
|
||||
}
|
||||
}
|
||||
|
||||
final class StartupTimingTests: XCTestCase {
|
||||
func testStartupTiming() throws {
|
||||
var t = Date()
|
||||
func lap(_ name: String) { print(String(format: "STARTUP %@: %.3fs", name, Date().timeIntervalSince(t))); t = Date() }
|
||||
_ = try Lexicon(contentsOf: resources.appendingPathComponent("lexicon.txt")); lap("lexicon")
|
||||
_ = EnglishModel(commonWordsURL: resources.appendingPathComponent("english-common.txt"),
|
||||
dictionaryURL: windowsData.appendingPathComponent("english-dict.txt"),
|
||||
bigramURL: windowsData.appendingPathComponent("english-bigram.json")); lap("english")
|
||||
_ = Features.loadWeights(from: resources.appendingPathComponent("classifier-weights.json")); lap("weights")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
#!/usr/bin/env python3
|
||||
"""IBus engine for AutoZhuyin (自動注音) on Linux.
|
||||
|
||||
All the logic lives in autozhuyin.core (shared with Windows and the tests); this file
|
||||
only translates IBus key events and draws the preedit / candidate table.
|
||||
|
||||
Run modes:
|
||||
autozhuyin_ibus.py --ibus started by ibus-daemon (from the component XML)
|
||||
autozhuyin_ibus.py [--activate] started by the user/autostart: registers the engine
|
||||
itself if ibus-daemon does not know it, and with
|
||||
--activate makes it the current input method
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
import gi # noqa: E402
|
||||
|
||||
gi.require_version("IBus", "1.0")
|
||||
from gi.repository import GLib, GObject, IBus # noqa: E402
|
||||
|
||||
import autozhuyin # noqa: E402
|
||||
from autozhuyin.core.classifier import LITERAL, Classifier, LearnedPreferences # noqa: E402
|
||||
from autozhuyin.core.engine import ( # noqa: E402
|
||||
BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, SHIFT_TAP, TAB, UP, CompositionEngine,
|
||||
)
|
||||
from autozhuyin.core.english import EnglishModel # noqa: E402
|
||||
from autozhuyin.core.lexicon import Lexicon # noqa: E402
|
||||
from autozhuyin.core.shift_tap import ShiftTapDetector # noqa: E402
|
||||
|
||||
ENGINE_NAME = "autozhuyin"
|
||||
BUS_NAME = "org.freedesktop.IBus.AutoZhuyin"
|
||||
VERSION = "0.0.0"
|
||||
|
||||
_DATA = os.path.join(os.path.dirname(os.path.abspath(autozhuyin.__file__)), "data")
|
||||
_USER_DIR = os.path.join(os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config"), "autozhuyin")
|
||||
_LEARNED_FILE = os.path.join(_USER_DIR, "learned.json")
|
||||
_USER_PHRASES = os.path.join(_USER_DIR, "user-phrases.txt")
|
||||
|
||||
try:
|
||||
with open(os.path.join(HERE, "version.txt"), encoding="utf-8") as _f:
|
||||
VERSION = _f.read().strip() or VERSION
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
_classifier = None
|
||||
|
||||
|
||||
def _save_learned(weights):
|
||||
try:
|
||||
os.makedirs(_USER_DIR, exist_ok=True)
|
||||
with open(_LEARNED_FILE, "w", encoding="utf-8") as f:
|
||||
json.dump(weights, f, ensure_ascii=False)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def shared_classifier():
|
||||
"""Loads the lexicon and English model once per process."""
|
||||
global _classifier
|
||||
if _classifier is None:
|
||||
lexicon = Lexicon.from_file(os.path.join(_DATA, "lexicon.txt"))
|
||||
if os.path.exists(_USER_PHRASES):
|
||||
with open(_USER_PHRASES, encoding="utf-8") as f:
|
||||
lexicon.load(f)
|
||||
english = EnglishModel.from_files(
|
||||
os.path.join(_DATA, "english-common.txt"),
|
||||
os.path.join(_DATA, "english-dict.txt"),
|
||||
os.path.join(_DATA, "english-bigram.json"),
|
||||
)
|
||||
weights = {}
|
||||
try:
|
||||
with open(_LEARNED_FILE, encoding="utf-8") as f:
|
||||
weights = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
trained = Classifier.load_weights(os.path.join(_DATA, "classifier-weights.json"))
|
||||
_classifier = Classifier(lexicon, english, LearnedPreferences(weights, _save_learned), trained)
|
||||
return _classifier
|
||||
|
||||
|
||||
_SPECIAL = {
|
||||
IBus.KEY_BackSpace: BACKSPACE, IBus.KEY_Return: ENTER, IBus.KEY_KP_Enter: ENTER,
|
||||
IBus.KEY_Escape: ESCAPE, IBus.KEY_Tab: TAB,
|
||||
IBus.KEY_Up: UP, IBus.KEY_Down: DOWN, IBus.KEY_Left: LEFT, IBus.KEY_Right: RIGHT,
|
||||
}
|
||||
_SHIFT_KEYS = (IBus.KEY_Shift_L, IBus.KEY_Shift_R)
|
||||
_SHORTCUT_MASK = (IBus.ModifierType.CONTROL_MASK | IBus.ModifierType.MOD1_MASK
|
||||
| IBus.ModifierType.SUPER_MASK | IBus.ModifierType.MOD4_MASK)
|
||||
_LITERAL_COLOR = 0x1E64DC # blue hint for English segments
|
||||
|
||||
|
||||
class AutoZhuyinEngine(IBus.Engine):
|
||||
__gtype_name__ = "AutoZhuyinEngine"
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self._setup()
|
||||
|
||||
def _setup(self):
|
||||
if getattr(self, "engine", None) is not None:
|
||||
return
|
||||
self.engine = CompositionEngine(shared_classifier())
|
||||
self.shift_tap = ShiftTapDetector()
|
||||
self.table = IBus.LookupTable.new(self.engine.page_size, 0, True, False)
|
||||
self.table.set_orientation(IBus.Orientation.VERTICAL)
|
||||
self.mode_prop = IBus.Property(
|
||||
key="mode", prop_type=IBus.PropType.NORMAL,
|
||||
label=IBus.Text.new_from_string("中英自動"),
|
||||
symbol=IBus.Text.new_from_string("自"),
|
||||
tooltip=IBus.Text.new_from_string("單按 Shift 切換英文模式"),
|
||||
sensitive=True, visible=True, state=IBus.PropState.UNCHECKED)
|
||||
self.props = IBus.PropList()
|
||||
self.props.append(self.mode_prop)
|
||||
|
||||
# --- keys ---
|
||||
|
||||
def do_process_key_event(self, keyval, keycode, state):
|
||||
self._setup()
|
||||
released = bool(state & IBus.ModifierType.RELEASE_MASK)
|
||||
|
||||
# Shift pressed and released alone: English mode (or flip the current word).
|
||||
if keyval in _SHIFT_KEYS:
|
||||
now = time.monotonic()
|
||||
if not released:
|
||||
if state & _SHORTCUT_MASK:
|
||||
self.shift_tap.key_pressed()
|
||||
else:
|
||||
self.shift_tap.shift_down(now)
|
||||
return False
|
||||
if self.shift_tap.shift_up(now):
|
||||
self.engine.handle(SHIFT_TAP)
|
||||
self._update(None)
|
||||
self._update_mode()
|
||||
return False
|
||||
if released:
|
||||
return False
|
||||
self.shift_tap.key_pressed()
|
||||
|
||||
# Shortcuts (Ctrl/Alt/Super) go to the app; commit any composition first.
|
||||
if state & _SHORTCUT_MASK:
|
||||
if self.engine.keys:
|
||||
self._update(self.engine.commit_all())
|
||||
return False
|
||||
|
||||
key = self._key(keyval, state)
|
||||
if key is None:
|
||||
return self._busy() # swallow other keys (Delete, Home…) while composing
|
||||
if not self.engine.wants(key):
|
||||
return False
|
||||
handled, commit = self.engine.handle(key)
|
||||
self._update(commit)
|
||||
return handled
|
||||
|
||||
def _key(self, keyval, state):
|
||||
if keyval in _SPECIAL:
|
||||
return _SPECIAL[keyval]
|
||||
if keyval == IBus.KEY_space:
|
||||
return " "
|
||||
code = IBus.keyval_to_unicode(keyval)
|
||||
if not code or not code.isprintable():
|
||||
return None
|
||||
# Caps Lock without Shift: keep Zhuyin working by using lowercase.
|
||||
if state & IBus.ModifierType.LOCK_MASK and not state & IBus.ModifierType.SHIFT_MASK:
|
||||
code = code.lower()
|
||||
return code
|
||||
|
||||
def _busy(self):
|
||||
return bool(self.engine.keys or self.engine.candidates)
|
||||
|
||||
# --- candidate panel (mouse) ---
|
||||
|
||||
def do_candidate_clicked(self, index, button, state):
|
||||
self.engine.select_candidate(self.engine.page_start + index)
|
||||
self._update(None)
|
||||
|
||||
def do_page_up(self):
|
||||
return self._move_candidate(-self.engine.page_size)
|
||||
|
||||
def do_page_down(self):
|
||||
return self._move_candidate(self.engine.page_size)
|
||||
|
||||
def do_cursor_up(self):
|
||||
return self._move_candidate(-1)
|
||||
|
||||
def do_cursor_down(self):
|
||||
return self._move_candidate(1)
|
||||
|
||||
def _move_candidate(self, delta):
|
||||
if not self.engine.candidates:
|
||||
return False
|
||||
e = self.engine
|
||||
e.candidate_index = max(0, min(len(e.candidates) - 1, e.candidate_index + delta))
|
||||
self._update(None)
|
||||
return True
|
||||
|
||||
# --- lifecycle ---
|
||||
|
||||
def do_focus_in(self):
|
||||
self._setup()
|
||||
self.register_properties(self.props)
|
||||
self._update_mode()
|
||||
|
||||
def do_focus_out(self):
|
||||
self._setup()
|
||||
# PreeditFocusMode.COMMIT: IBus commits the shown text itself on focus out.
|
||||
self.engine.reset()
|
||||
self.hide_lookup_table()
|
||||
|
||||
def do_reset(self):
|
||||
self._setup()
|
||||
self.engine.reset()
|
||||
self.hide_preedit_text()
|
||||
self.hide_lookup_table()
|
||||
|
||||
def do_disable(self):
|
||||
self.do_reset()
|
||||
|
||||
# --- drawing ---
|
||||
|
||||
def _update(self, commit):
|
||||
if commit:
|
||||
self.commit_text(IBus.Text.new_from_string(commit))
|
||||
pieces = self.engine.marked_pieces
|
||||
text = "".join(t for t, _ in pieces)
|
||||
if text:
|
||||
ibus_text = IBus.Text.new_from_string(text)
|
||||
attrs = IBus.AttrList()
|
||||
attrs.append(IBus.attr_underline_new(IBus.AttrUnderline.SINGLE, 0, len(text)))
|
||||
pos = 0
|
||||
for piece, mode in pieces:
|
||||
if mode == LITERAL and piece:
|
||||
attrs.append(IBus.attr_foreground_new(_LITERAL_COLOR, pos, pos + len(piece)))
|
||||
pos += len(piece)
|
||||
ibus_text.set_attributes(attrs)
|
||||
self.update_preedit_text_with_mode(ibus_text, self.engine.cursor_offset, True,
|
||||
IBus.PreeditFocusMode.COMMIT)
|
||||
else:
|
||||
self.hide_preedit_text()
|
||||
|
||||
if self.engine.candidates:
|
||||
self.table.clear()
|
||||
for _, candidate in self.engine.candidates:
|
||||
self.table.append_candidate(IBus.Text.new_from_string(candidate))
|
||||
self.table.set_cursor_pos(self.engine.candidate_index)
|
||||
self.update_lookup_table(self.table, True)
|
||||
else:
|
||||
self.hide_lookup_table()
|
||||
|
||||
def _update_mode(self):
|
||||
english = self.engine.english_mode
|
||||
self.mode_prop.set_label(IBus.Text.new_from_string("英文" if english else "中英自動"))
|
||||
self.mode_prop.set_symbol(IBus.Text.new_from_string("英" if english else "自"))
|
||||
self.update_property(self.mode_prop)
|
||||
|
||||
|
||||
# --- process setup ---
|
||||
|
||||
def make_component(exec_path):
|
||||
component = IBus.Component(
|
||||
name=BUS_NAME, description="自動注音 AutoZhuyin", version=VERSION,
|
||||
license="Private", author="AutoZhuyin", homepage="",
|
||||
command_line=f"{sys.executable} {exec_path} --ibus", textdomain="")
|
||||
icon = os.path.join(HERE, "icon.png")
|
||||
component.add_engine(IBus.EngineDesc(
|
||||
name=ENGINE_NAME, longname="自動注音", description="大千注音+英文,自動判斷中英",
|
||||
language="zh_TW", license="Private", author="AutoZhuyin",
|
||||
icon=icon if os.path.exists(icon) else "", layout="us", symbol="自", rank=0))
|
||||
return component
|
||||
|
||||
|
||||
def main(argv):
|
||||
IBus.init()
|
||||
bus = IBus.Bus()
|
||||
loop = GLib.MainLoop()
|
||||
started_by_ibus = "--ibus" in argv
|
||||
activate = "--activate" in argv
|
||||
factory = None
|
||||
|
||||
def on_connected(*_):
|
||||
nonlocal factory
|
||||
factory = IBus.Factory.new(bus.get_connection())
|
||||
factory.add_engine(ENGINE_NAME, GObject.type_from_name("AutoZhuyinEngine"))
|
||||
if started_by_ibus:
|
||||
bus.request_name(BUS_NAME, 0)
|
||||
else:
|
||||
known = any(e.get_name() == ENGINE_NAME for e in bus.list_engines())
|
||||
if not known:
|
||||
bus.register_component(make_component(os.path.abspath(__file__)))
|
||||
if activate:
|
||||
bus.set_global_engine(ENGINE_NAME)
|
||||
|
||||
def on_disconnected(*_):
|
||||
if started_by_ibus:
|
||||
loop.quit()
|
||||
# Otherwise wait: IBus.Bus reconnects (and emits "connected") when ibus-daemon restarts.
|
||||
|
||||
bus.connect("connected", on_connected)
|
||||
bus.connect("disconnected", on_disconnected)
|
||||
shared_classifier() # load data before the first key
|
||||
if bus.is_connected():
|
||||
on_connected()
|
||||
elif started_by_ibus:
|
||||
print("autozhuyin: cannot connect to ibus-daemon", file=sys.stderr)
|
||||
return 1
|
||||
loop.run()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,57 @@
|
||||
自動注音 AutoZhuyin(Linux 測試版,IBus)
|
||||
=========================================
|
||||
|
||||
用大千注音鍵盤直接打字,不用切換中英文,輸入法會自動判斷你打的是注音還是英文。
|
||||
|
||||
按鍵: ji3 hello world su3cl3 (最後按空白)
|
||||
輸出: 我hello world 你好
|
||||
|
||||
適用:Bazzite、SteamOS 桌面模式、Fedora 等有 IBus 的系統(KDE 或 GNOME)。
|
||||
只在桌面模式可用;Steam 遊戲模式不支援系統輸入法。
|
||||
|
||||
|
||||
安裝
|
||||
----
|
||||
1. 解壓縮,在終端機(Konsole)進到這個資料夾,執行:
|
||||
./install.sh
|
||||
不需要 sudo,檔案只會裝在你的家目錄(~/.local/share/autozhuyin)。
|
||||
|
||||
2. KDE(Bazzite / SteamOS 預設):
|
||||
系統設定 → 鍵盤 → 虛擬鍵盤,選「IBus Wayland」→ 套用,然後登出再登入。
|
||||
GNOME:IBus 是內建的,不用另外設定。
|
||||
|
||||
3. 之後每次登入都會自動切換到自動注音。
|
||||
|
||||
|
||||
使用方式
|
||||
--------
|
||||
空白 注音:一聲;音節打完時送出中文。英文:送出單字加空白
|
||||
Enter 送出目前組字
|
||||
Tab 判斷錯了?按 Tab 在中/英之間翻轉,輸入法會記住這次修正
|
||||
← → 在組字區移動游標;游標在中間時直接打注音會插入新字,
|
||||
Backspace 刪掉游標前的字
|
||||
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字
|
||||
Esc 取消組字
|
||||
單按 Shift 切換英文模式(按鍵直接送出),再單按一次回到中英自動判斷
|
||||
Shift+字母 打大寫字母(也會強制英文)
|
||||
` + 標點 強制全形標點,例如 `< 或 `, = , `. = 。 `/ = ? `; = ;
|
||||
|
||||
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
|
||||
|
||||
|
||||
切換與移除
|
||||
----------
|
||||
暫時改回一般鍵盤: ibus engine xkb:us::eng
|
||||
切回自動注音: ibus engine autozhuyin
|
||||
移除: ./uninstall.sh
|
||||
|
||||
個人資料(學習紀錄、自訂詞)在 ~/.config/autozhuyin/:
|
||||
learned.json 按 Tab 修正的學習紀錄
|
||||
user-phrases.txt 自訂詞,一行一個,例如:ㄗˋ-ㄉㄨㄥˋ 自動 -1
|
||||
|
||||
|
||||
有問題時
|
||||
--------
|
||||
在終端機手動執行看看錯誤訊息:
|
||||
/usr/bin/python3 ~/.local/share/autozhuyin/autozhuyin_ibus.py --activate
|
||||
把畫面上的訊息回報給我。
|
||||
Executable
+63
@@ -0,0 +1,63 @@
|
||||
#!/bin/bash
|
||||
# Installs AutoZhuyin (自動注音) for the current user as an IBus input method.
|
||||
# Nothing outside your home folder is touched, so it works on read-only systems
|
||||
# (Bazzite, SteamOS, Fedora Atomic). No sudo needed.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
DEST="${XDG_DATA_HOME:-$HOME/.local/share}/autozhuyin"
|
||||
AUTOSTART="${XDG_CONFIG_HOME:-$HOME/.config}/autostart"
|
||||
PY=/usr/bin/python3
|
||||
|
||||
echo "== 自動注音 AutoZhuyin 安裝 =="
|
||||
|
||||
# 1. IBus and its Python bindings must come with the system.
|
||||
if ! command -v ibus >/dev/null 2>&1; then
|
||||
echo "找不到 IBus(ibus 指令)。這個系統沒有內建 IBus,自動注音目前只支援 IBus。" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! "$PY" -c 'import gi; gi.require_version("IBus", "1.0"); from gi.repository import IBus' 2>/dev/null; then
|
||||
echo "系統的 Python 缺少 IBus 模組(python3-gobject / IBus-1.0 typelib)。" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 2. Copy the program.
|
||||
pkill -f "autozhuyin/autozhuyin_ibus.py" 2>/dev/null || true
|
||||
rm -rf "$DEST"
|
||||
mkdir -p "$DEST"
|
||||
cp -R autozhuyin autozhuyin_ibus.py version.txt "$DEST/"
|
||||
[ -f icon.png ] && cp icon.png "$DEST/"
|
||||
[ -f THIRD_PARTY_LICENSES.txt ] && cp THIRD_PARTY_LICENSES.txt "$DEST/"
|
||||
chmod +x "$DEST/autozhuyin_ibus.py"
|
||||
|
||||
# 3. Start with the desktop session: registers the engine with IBus and selects it.
|
||||
mkdir -p "$AUTOSTART"
|
||||
cat > "$AUTOSTART/autozhuyin.desktop" <<DESKTOP
|
||||
[Desktop Entry]
|
||||
Type=Application
|
||||
Name=自動注音 AutoZhuyin
|
||||
Comment=Registers the AutoZhuyin input method with IBus
|
||||
Exec=$PY $DEST/autozhuyin_ibus.py --activate
|
||||
Icon=$DEST/icon.png
|
||||
X-GNOME-Autostart-enabled=true
|
||||
X-KDE-autostart-phase=2
|
||||
NoDisplay=true
|
||||
DESKTOP
|
||||
|
||||
# 4. Start it now as well.
|
||||
if pgrep -x ibus-daemon >/dev/null 2>&1; then
|
||||
nohup "$PY" "$DEST/autozhuyin_ibus.py" --activate >/dev/null 2>&1 &
|
||||
sleep 1
|
||||
echo "已啟動並切換到自動注音。"
|
||||
else
|
||||
echo "IBus 目前沒有在執行,請照下面的步驟啟用 IBus 後登出再登入。"
|
||||
fi
|
||||
|
||||
cat <<'MSG'
|
||||
|
||||
安裝完成。
|
||||
- KDE(Bazzite / SteamOS 桌面模式):系統設定 → 鍵盤 → 虛擬鍵盤,選「IBus Wayland」,
|
||||
按套用後登出再登入。之後每次登入都會自動切換到自動注音。
|
||||
- 想暫時改回一般鍵盤:ibus engine xkb:us::eng ;切回自動注音:ibus engine autozhuyin
|
||||
- 移除:執行 uninstall.sh
|
||||
MSG
|
||||
Executable
+14
@@ -0,0 +1,14 @@
|
||||
#!/bin/bash
|
||||
# Removes AutoZhuyin (自動注音) for the current user. Your learned corrections and
|
||||
# user phrases in ~/.config/autozhuyin are kept (delete that folder to remove them).
|
||||
set -uo pipefail
|
||||
|
||||
DEST="${XDG_DATA_HOME:-$HOME/.local/share}/autozhuyin"
|
||||
AUTOSTART="${XDG_CONFIG_HOME:-$HOME/.config}/autostart/autozhuyin.desktop"
|
||||
|
||||
if command -v ibus >/dev/null 2>&1 && [ "$(ibus engine 2>/dev/null)" = "autozhuyin" ]; then
|
||||
ibus engine xkb:us::eng 2>/dev/null
|
||||
fi
|
||||
pkill -f "autozhuyin/autozhuyin_ibus.py" 2>/dev/null
|
||||
rm -rf "$DEST" "$AUTOSTART"
|
||||
echo "已移除自動注音。學習紀錄留在 ~/.config/autozhuyin,不需要可以自行刪除。"
|
||||
@@ -0,0 +1,167 @@
|
||||
"""Drives autozhuyin_ibus.AutoZhuyinEngine through a minimal stand-in for
|
||||
gi.repository.IBus, to catch wrapper bugs without Linux. Run from the repo root:
|
||||
python3 -m unittest discover -s linux/tests
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
|
||||
LINUX = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(LINUX), "windows")) # autozhuyin package
|
||||
sys.path.insert(0, LINUX)
|
||||
|
||||
|
||||
# --- stand-in for gi / IBus (only what the wrapper uses) ---
|
||||
class Flag(int):
|
||||
def __or__(self, o): return Flag(int(self) | int(o))
|
||||
def __and__(self, o): return Flag(int(self) & int(o))
|
||||
|
||||
|
||||
IBus = types.SimpleNamespace()
|
||||
for i, name in enumerate(["BackSpace", "Return", "KP_Enter", "Escape", "Tab", "Up", "Down", "Left",
|
||||
"Right", "space", "Shift_L", "Shift_R", "Delete"]):
|
||||
setattr(IBus, "KEY_" + name, 0xff00 + i)
|
||||
IBus.KEY_space = 0x20
|
||||
IBus.ModifierType = types.SimpleNamespace(
|
||||
SHIFT_MASK=Flag(1), LOCK_MASK=Flag(2), CONTROL_MASK=Flag(4), MOD1_MASK=Flag(8),
|
||||
SUPER_MASK=Flag(1 << 26), MOD4_MASK=Flag(64), RELEASE_MASK=Flag(1 << 30))
|
||||
IBus.keyval_to_unicode = lambda k: chr(k) if 0x20 <= k < 0x7f else "\x00"
|
||||
|
||||
|
||||
class Text:
|
||||
def __init__(self, s): self.text, self.attrs = s, None
|
||||
@staticmethod
|
||||
def new_from_string(s): return Text(s)
|
||||
def set_attributes(self, a): self.attrs = a
|
||||
|
||||
|
||||
class AttrList(list):
|
||||
pass
|
||||
|
||||
|
||||
class LookupTable:
|
||||
def __init__(self): self.items, self.cursor = [], 0
|
||||
@staticmethod
|
||||
def new(*a): return LookupTable()
|
||||
def set_orientation(self, o): pass
|
||||
def clear(self): self.items = []
|
||||
def append_candidate(self, t): self.items.append(t.text)
|
||||
def set_cursor_pos(self, p): self.cursor = p
|
||||
|
||||
|
||||
class Property:
|
||||
def __init__(self, **kw): self.label = kw["label"].text
|
||||
def set_label(self, t): self.label = t.text
|
||||
def set_symbol(self, t): pass
|
||||
|
||||
|
||||
class Engine:
|
||||
def __init__(self, *a, **kw):
|
||||
self.committed, self.preedit, self.cursor, self.shown = "", "", 0, None
|
||||
self.preedit_attrs = None
|
||||
|
||||
def commit_text(self, t): self.committed += t.text
|
||||
def update_preedit_text_with_mode(self, t, cursor, visible, mode):
|
||||
self.preedit, self.cursor, self.preedit_attrs = t.text, cursor, t.attrs
|
||||
def hide_preedit_text(self): self.preedit = ""
|
||||
def update_lookup_table(self, table, visible): self.shown = list(table.items)
|
||||
def hide_lookup_table(self): self.shown = None
|
||||
def register_properties(self, p): pass
|
||||
def update_property(self, p): self.mode_label = p.label
|
||||
|
||||
|
||||
IBus.Text, IBus.AttrList, IBus.LookupTable, IBus.Property, IBus.Engine = Text, AttrList, LookupTable, Property, Engine
|
||||
IBus.PropList = lambda: types.SimpleNamespace(append=lambda p: None)
|
||||
IBus.attr_underline_new = lambda kind, a, b: ("underline", a, b)
|
||||
IBus.attr_foreground_new = lambda color, a, b: ("color", a, b)
|
||||
IBus.AttrUnderline = types.SimpleNamespace(SINGLE=1)
|
||||
IBus.Orientation = types.SimpleNamespace(VERTICAL=1)
|
||||
IBus.PropType = types.SimpleNamespace(NORMAL=0)
|
||||
IBus.PropState = types.SimpleNamespace(UNCHECKED=0)
|
||||
IBus.PreeditFocusMode = types.SimpleNamespace(COMMIT=1)
|
||||
|
||||
gi = types.ModuleType("gi")
|
||||
gi.require_version = lambda *a: None
|
||||
repo = types.ModuleType("gi.repository")
|
||||
repo.IBus, repo.GLib, repo.GObject = IBus, types.SimpleNamespace(), types.SimpleNamespace()
|
||||
gi.repository = repo
|
||||
sys.modules["gi"], sys.modules["gi.repository"] = gi, repo
|
||||
|
||||
import autozhuyin_ibus # noqa: E402
|
||||
|
||||
M = IBus.ModifierType
|
||||
|
||||
|
||||
def press(e, keys):
|
||||
"""Types keys; returns the list of 'handled' results."""
|
||||
names = {"\n": IBus.KEY_Return, "\b": IBus.KEY_BackSpace, "\t": IBus.KEY_Tab, "←": IBus.KEY_Left,
|
||||
"→": IBus.KEY_Right, "↓": IBus.KEY_Down, " ": IBus.KEY_space}
|
||||
out = []
|
||||
for c in keys:
|
||||
kv = names.get(c, ord(c))
|
||||
out.append(e.do_process_key_event(kv, 0, Flag(0)))
|
||||
e.do_process_key_event(kv, 0, M.RELEASE_MASK)
|
||||
return out
|
||||
|
||||
|
||||
class IBusEngineTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.e = autozhuyin_ibus.AutoZhuyinEngine()
|
||||
|
||||
def test_mixed_sentence(self):
|
||||
press(self.e, "ji3 hello world su3cl3 ")
|
||||
self.assertEqual(self.e.committed, "我hello world 你好")
|
||||
self.assertEqual(self.e.preedit, "")
|
||||
|
||||
def test_preedit_marks_english_blue(self):
|
||||
press(self.e, "ji3hello")
|
||||
self.assertEqual(self.e.preedit, "我hello")
|
||||
self.assertIn(("color", 1, 6), self.e.preedit_attrs)
|
||||
self.assertEqual(self.e.cursor, 6)
|
||||
|
||||
def test_space_passes_through_when_idle(self):
|
||||
self.assertEqual(press(self.e, " "), [False])
|
||||
|
||||
def test_candidates_and_click(self):
|
||||
press(self.e, "su3↓")
|
||||
self.assertTrue(self.e.shown)
|
||||
second = self.e.shown[1]
|
||||
self.e.do_candidate_clicked(1, 1, 0)
|
||||
self.assertIsNone(self.e.shown)
|
||||
press(self.e, "\n")
|
||||
self.assertEqual(self.e.committed, second)
|
||||
|
||||
def test_shift_tap_toggles_english(self):
|
||||
self.e.do_process_key_event(IBus.KEY_Shift_L, 0, Flag(0))
|
||||
self.e.do_process_key_event(IBus.KEY_Shift_L, 0, M.SHIFT_MASK | M.RELEASE_MASK)
|
||||
self.assertEqual(self.e.mode_label, "英文")
|
||||
self.assertEqual(press(self.e, "a"), [False]) # goes to the app as typed
|
||||
|
||||
def test_shift_as_modifier_is_not_a_tap(self):
|
||||
self.e.do_process_key_event(IBus.KEY_Shift_L, 0, Flag(0))
|
||||
self.e.do_process_key_event(ord("A"), 0, M.SHIFT_MASK)
|
||||
self.e.do_process_key_event(IBus.KEY_Shift_L, 0, M.SHIFT_MASK | M.RELEASE_MASK)
|
||||
self.assertFalse(self.e.engine.english_mode)
|
||||
|
||||
def test_ctrl_shortcut_commits_and_passes(self):
|
||||
press(self.e, "su3")
|
||||
self.assertFalse(self.e.do_process_key_event(ord("c"), 0, M.CONTROL_MASK))
|
||||
self.assertEqual(self.e.committed, "你")
|
||||
|
||||
def test_backtick_symbol(self):
|
||||
press(self.e, "`<")
|
||||
self.assertEqual(self.e.committed, ",")
|
||||
|
||||
def test_insert_in_middle(self):
|
||||
press(self.e, "rup fu4←wu0 \n")
|
||||
self.assertEqual(self.e.committed, "今天氣")
|
||||
|
||||
def test_focus_out_resets(self):
|
||||
press(self.e, "su3")
|
||||
self.e.do_focus_out()
|
||||
self.assertEqual(self.e.engine.keys, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -19,6 +19,10 @@ mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
|
||||
cp "$BIN" "$APP/Contents/MacOS/AutoZhuyin"
|
||||
cp Resources/Info.plist "$APP/Contents/Info.plist"
|
||||
cp Resources/lexicon.txt Resources/english-common.txt Resources/classifier-weights.json "$APP/Contents/Resources/"
|
||||
cp THIRD_PARTY_LICENSES.txt "$APP/Contents/Resources/"
|
||||
# Precomputed English dictionary and bigram table (shared with the Windows build).
|
||||
python3 -I windows/tools/build_data.py >/dev/null
|
||||
cp windows/autozhuyin/data/english-dict.txt windows/autozhuyin/data/english-bigram.json "$APP/Contents/Resources/"
|
||||
|
||||
if [ ! -f build/icon.tiff ]; then
|
||||
xcrun swift scripts/make-icon.swift build/icon.tiff
|
||||
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/bin/bash
|
||||
# Builds build/AutoZhuyin-Linux-<version>.tar.gz: the IBus engine, the shared Python
|
||||
# core and data, and install/uninstall scripts (per-user install, no root needed).
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
VERSION="$(python3 -I -c 'import json,sys; print(json.load(open(sys.argv[1]))["version"])' windows/autozhuyin/ime.json)"
|
||||
NAME="AutoZhuyin-Linux-$VERSION"
|
||||
OUT="build/$NAME"
|
||||
|
||||
python3 -I windows/tools/build_data.py
|
||||
(cd windows && python3 -m unittest discover -q tests)
|
||||
python3 -m unittest discover -q -s linux/tests
|
||||
|
||||
if [ ! -f build/icon.tiff ]; then
|
||||
DEVELOPER_DIR="${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}" xcrun swift scripts/make-icon.swift build/icon.tiff
|
||||
fi
|
||||
|
||||
rm -rf "$OUT" "$OUT.tar.gz"
|
||||
mkdir -p "$OUT/autozhuyin"
|
||||
rsync -a --exclude __pycache__ --exclude '*.pyc' \
|
||||
windows/autozhuyin/__init__.py windows/autozhuyin/core windows/autozhuyin/data "$OUT/autozhuyin/"
|
||||
cp linux/autozhuyin_ibus.py linux/installer/install.sh linux/installer/uninstall.sh linux/installer/README.txt \
|
||||
THIRD_PARTY_LICENSES.txt "$OUT/"
|
||||
echo "$VERSION" > "$OUT/version.txt"
|
||||
sips -s format png -z 64 64 build/icon.tiff --out "$OUT/icon.png" >/dev/null
|
||||
chmod +x "$OUT/install.sh" "$OUT/uninstall.sh" "$OUT/autozhuyin_ibus.py"
|
||||
|
||||
# Plain ustar without macOS extended attributes, so GNU tar doesn't warn.
|
||||
(cd build && COPYFILE_DISABLE=1 tar --no-xattrs --format ustar -czf "$NAME.tar.gz" "$NAME")
|
||||
echo "Built build/$NAME.tar.gz"
|
||||
@@ -26,6 +26,9 @@ rsync -a --exclude __pycache__ --exclude '*.pyc' windows/autozhuyin "$OUT/"
|
||||
for f in install.bat uninstall.bat README.txt; do
|
||||
sed 's/$/\r/' "windows/installer/$f" > "$OUT/$f"
|
||||
done
|
||||
# Third-party notices: next to the installer and inside the installed folder.
|
||||
sed 's/$/\r/' THIRD_PARTY_LICENSES.txt > "$OUT/THIRD_PARTY_LICENSES.txt"
|
||||
cp "$OUT/THIRD_PARTY_LICENSES.txt" "$OUT/autozhuyin/"
|
||||
|
||||
(cd build && zip -qr -X "$NAME.zip" "$NAME")
|
||||
echo "Built build/$NAME.zip"
|
||||
|
||||
+5
-3
@@ -1,5 +1,5 @@
|
||||
#!/bin/bash
|
||||
# One-step release: bump the version, run all tests, build both installers, then
|
||||
# One-step release: bump the version, run all tests, build all installers, then
|
||||
# commit the version change and tag it.
|
||||
#
|
||||
# Usage: scripts/release.sh [patch|minor|major] [--push] (default: patch)
|
||||
@@ -38,14 +38,16 @@ xcrun swift test -q 2>&1 | tail -3
|
||||
echo "==> Packaging (runs the Python tests too)"
|
||||
scripts/package.sh >/dev/null
|
||||
scripts/package-windows.sh >/dev/null
|
||||
scripts/package-linux.sh >/dev/null
|
||||
|
||||
trap - ERR
|
||||
git add Resources/Info.plist windows/autozhuyin/ime.json
|
||||
git commit -q -m "Release v$VERSION"
|
||||
git tag -a "v$VERSION" -m "AutoZhuyin v$VERSION"
|
||||
echo "==> Tagged v$VERSION"
|
||||
ls -lh "build/AutoZhuyin-$VERSION.pkg" "build/AutoZhuyin-Windows-$VERSION.zip"
|
||||
shasum -a 256 "build/AutoZhuyin-$VERSION.pkg" "build/AutoZhuyin-Windows-$VERSION.zip"
|
||||
FILES=("build/AutoZhuyin-$VERSION.pkg" "build/AutoZhuyin-Windows-$VERSION.zip" "build/AutoZhuyin-Linux-$VERSION.tar.gz")
|
||||
ls -lh "${FILES[@]}"
|
||||
shasum -a 256 "${FILES[@]}"
|
||||
|
||||
if [ "$PUSH" = 1 ]; then
|
||||
git push origin HEAD "v$VERSION"
|
||||
|
||||
@@ -23,14 +23,14 @@ def main():
|
||||
clf = core.Classifier(lexicon, english, weights=core.Classifier.load_weights(WEIGHTS_PATH))
|
||||
gen = Generator(os.path.join(DATA, "lexicon.txt"), os.path.join(DATA, "english-common.txt"),
|
||||
os.path.join(DATA, "english-dict.txt"), seed=7)
|
||||
cases = [(k, f) for k in HAND_PICKED for f in (False, True)]
|
||||
while len(cases) < 400:
|
||||
cases = [(k, f, p) for k in HAND_PICKED for f in (False, True) for p in (None, "literal")]
|
||||
while len(cases) < 500:
|
||||
ex = gen.example()
|
||||
if len(ex["keys"]) <= 28:
|
||||
cases.append((ex["keys"], ex["finishing"]))
|
||||
cases.append((gen.prefix_of(ex)["keys"], False))
|
||||
fixtures = [{"keys": k, "finishing": f,
|
||||
"segments": [[s, e, m] for s, e, m in clf.segment(list(k), f)]} for k, f in cases]
|
||||
cases.append((ex["keys"], ex["finishing"], ex["previous"]))
|
||||
cases.append((gen.prefix_of(ex)["keys"], False, ex["previous"]))
|
||||
fixtures = [{"keys": k, "finishing": f, "previous": p,
|
||||
"segments": [[s, e, m] for s, e, m in clf.segment(list(k), f, p)]} for k, f, p in cases]
|
||||
with open(OUT, "w", encoding="utf-8") as fh:
|
||||
json.dump(fixtures, fh, ensure_ascii=False, indent=0)
|
||||
print("wrote %d fixtures to %s" % (len(fixtures), OUT))
|
||||
|
||||
+5114
-3374
File diff suppressed because it is too large
Load Diff
+17
-2
@@ -102,16 +102,31 @@ class Generator:
|
||||
labels = "".join(m * len(p) for m, p in parts)
|
||||
if parts[-1][0] == "l":
|
||||
finishing = True # English / numbers end with a space
|
||||
elif keys[-1] == " " and self.rng.random() < 0.6:
|
||||
# The moment the space (first tone) is pressed: the buffer has no space yet and
|
||||
# the input method decides "first tone" vs "English word + space".
|
||||
keys, labels, finishing = keys[:-1], labels[:-1], True
|
||||
kind += "/tone1"
|
||||
else:
|
||||
# Chinese ends with Enter, or space after a finished (toned) syllable.
|
||||
finishing = keys[-1] != " " and self.rng.random() < 0.5
|
||||
return {"keys": keys, "labels": labels, "finishing": finishing, "kind": kind}
|
||||
return {"keys": keys, "labels": labels, "finishing": finishing, "kind": kind,
|
||||
"previous": self.previous_mode(labels[0])}
|
||||
|
||||
def previous_mode(self, first_label):
|
||||
"""Mode of the text committed just before: none, mostly the same, sometimes the other."""
|
||||
r = self.rng.random()
|
||||
if r < 0.35:
|
||||
return None
|
||||
same = core.ZHUYIN if first_label == "z" else core.LITERAL
|
||||
other = core.LITERAL if first_label == "z" else core.ZHUYIN
|
||||
return same if r < 0.80 else other
|
||||
|
||||
def prefix_of(self, ex):
|
||||
"""The buffer while still typing: a random cut, not finishing."""
|
||||
cut = self.rng.randint(1, len(ex["keys"]) - 1) if len(ex["keys"]) > 1 else 1
|
||||
return {"keys": ex["keys"][:cut], "labels": ex["labels"][:cut], "finishing": False,
|
||||
"kind": ex["kind"] + "/prefix"}
|
||||
"kind": ex["kind"] + "/prefix", "previous": ex["previous"]}
|
||||
|
||||
|
||||
def _cumulative(ws):
|
||||
|
||||
+19
-10
@@ -45,12 +45,12 @@ def labels_of(segs, n):
|
||||
return "".join(lab)
|
||||
|
||||
|
||||
def seg_features(clf, keys, segs, finishing):
|
||||
def seg_features(clf, keys, segs, finishing, previous):
|
||||
"""Summed features of a segmentation, or None if any segment is impossible."""
|
||||
total = defaultdict(float)
|
||||
n = len(keys)
|
||||
for idx, (s, e, m) in enumerate(segs):
|
||||
fz, fl = clf.segment_features(keys[s:e], e == n, idx > 0, finishing)
|
||||
fz, fl = clf.segment_features(keys[s:e], e == n, idx > 0, finishing, previous if idx == 0 else None)
|
||||
f = fz if m == core.ZHUYIN else fl
|
||||
if f is None:
|
||||
return None
|
||||
@@ -62,12 +62,13 @@ def seg_features(clf, keys, segs, finishing):
|
||||
def evaluate(clf, weights, data):
|
||||
by_kind = defaultdict(lambda: [0, 0])
|
||||
for ex in data:
|
||||
segs = clf.segment(list(ex["keys"]), ex["finishing"], None, weights)
|
||||
segs = clf.segment(list(ex["keys"]), ex["finishing"], ex["previous"], weights)
|
||||
ok = labels_of(segs, len(ex["keys"])) == ex["labels"]
|
||||
bucket = by_kind["prefix" if ex["kind"].endswith("/prefix") else ex["kind"]]
|
||||
bucket[0] += ok
|
||||
bucket[1] += 1
|
||||
commit = [v for k, v in by_kind.items() if k != "prefix"]
|
||||
kind = "prefix" if ex["kind"].endswith("/prefix") else ex["kind"]
|
||||
for k in (kind, "after-other") if _after_other(ex) else (kind,):
|
||||
by_kind[k][0] += ok
|
||||
by_kind[k][1] += 1
|
||||
commit = [v for k, v in by_kind.items() if k not in ("prefix", "after-other")]
|
||||
summary = {
|
||||
"commit": sum(c for c, _ in commit) / max(1, sum(t for _, t in commit)),
|
||||
"prefix": by_kind["prefix"][0] / max(1, by_kind["prefix"][1]),
|
||||
@@ -76,6 +77,14 @@ def evaluate(clf, weights, data):
|
||||
return summary
|
||||
|
||||
|
||||
def _after_other(ex):
|
||||
"""Committed buffer whose first segment differs from the text typed just before."""
|
||||
if ex["kind"].endswith("/prefix") or ex["previous"] is None:
|
||||
return False
|
||||
first = core.ZHUYIN if ex["labels"][0] == "z" else core.LITERAL
|
||||
return ex["previous"] != first
|
||||
|
||||
|
||||
def show(name, s):
|
||||
kinds = " ".join("%s %.1f%%" % (k, 100 * v) for k, v in s["by_kind"].items())
|
||||
print("%-9s commit %.2f%% typing %.2f%% [%s]" % (name, 100 * s["commit"], 100 * s["prefix"], kinds))
|
||||
@@ -92,11 +101,11 @@ def train(clf, data, epochs, lr, seed):
|
||||
t0 = time.time()
|
||||
for ex in data:
|
||||
keys = list(ex["keys"])
|
||||
pred = clf.segment(keys, ex["finishing"], None, w)
|
||||
pred = clf.segment(keys, ex["finishing"], ex["previous"], w)
|
||||
if labels_of(pred, len(keys)) != ex["labels"]:
|
||||
gold = gold_segments(ex["labels"])
|
||||
fg = seg_features(clf, keys, gold, ex["finishing"])
|
||||
fp = seg_features(clf, keys, pred, ex["finishing"])
|
||||
fg = seg_features(clf, keys, gold, ex["finishing"], ex["previous"])
|
||||
fp = seg_features(clf, keys, pred, ex["finishing"], ex["previous"])
|
||||
if fg is not None and fp is not None:
|
||||
mistakes += 1
|
||||
scale = lr * (0.5 if ex["kind"].endswith("/prefix") else 1.0)
|
||||
|
||||
@@ -142,8 +142,8 @@ class AutoZhuyinTextService(TextService):
|
||||
return False
|
||||
self.engine.handle(SHIFT_TAP)
|
||||
self._update(None)
|
||||
if self.engine.english_armed:
|
||||
self.showMessage("英文", 1)
|
||||
if not self.engine.keys:
|
||||
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
|
||||
return True
|
||||
|
||||
# --- replies ---
|
||||
|
||||
@@ -30,7 +30,6 @@ class LearnedPreferences:
|
||||
|
||||
|
||||
class Classifier:
|
||||
context_weight = 1.5
|
||||
|
||||
def __init__(self, lexicon, english, learned=None, weights=None):
|
||||
self.lexicon = lexicon
|
||||
@@ -49,20 +48,17 @@ class Classifier:
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
|
||||
def segment_features(self, seg, is_last, after_other, finishing):
|
||||
"""(zhuyin features or None, literal features or None) for one segment."""
|
||||
def segment_features(self, seg, is_last, after_other, finishing, previous=None):
|
||||
"""(zhuyin features or None, literal features or None) for one segment.
|
||||
`previous` (mode of the text committed just before) only affects the first segment."""
|
||||
seg = list(seg)
|
||||
return (zhuyin_features(seg, self.lexicon, is_last, after_other, finishing),
|
||||
literal_features(seg, self.english, is_last, after_other, finishing))
|
||||
return (zhuyin_features(seg, self.lexicon, is_last, after_other, finishing, previous),
|
||||
literal_features(seg, self.english, is_last, after_other, finishing, previous))
|
||||
|
||||
def _scores(self, seg, is_last, after_other, finishing, previous, whole, weights):
|
||||
fz, fl = self.segment_features(seg, is_last, after_other, finishing)
|
||||
fz, fl = self.segment_features(seg, is_last, after_other, finishing, previous)
|
||||
zh = dot(weights, fz) if fz is not None else NEG_INF
|
||||
lit = dot(weights, fl) if fl is not None else NEG_INF
|
||||
if previous == ZHUYIN:
|
||||
zh += self.context_weight
|
||||
elif previous == LITERAL:
|
||||
lit += self.context_weight
|
||||
if whole:
|
||||
b = self.learned.bias("".join(seg))
|
||||
if b > 0:
|
||||
|
||||
@@ -20,6 +20,19 @@ FULLWIDTH = {
|
||||
"{": "『", "}": "』", "[": "「", "]": "」", "~": "~", "_": "——",
|
||||
}
|
||||
|
||||
# An English word that is also a common one-syllable Chinese reading ("up " = ㄧㄣ 因)
|
||||
# is not committed on space; the next word decides ("up jo4" = 因為, "up the" = up the).
|
||||
HOLD_MIN_SCORE = -3.5
|
||||
|
||||
# "`" then one of these keys always gives the full-width symbol, whatever the context.
|
||||
# "`" followed by anything else is just a backtick (so ``` still works).
|
||||
BACKTICK = "`"
|
||||
SYMBOLS = {
|
||||
",": ",", "<": ",", ".": "。", ">": "。", "/": "?", "?": "?", ";": ";", ":": ":",
|
||||
"'": "、", "\\": "、", '"': "「", "[": "「", "]": "」", "{": "『", "}": "』",
|
||||
"(": "(", ")": ")", "!": "!", "~": "~", "-": "——", "_": "——", "=": "=",
|
||||
}
|
||||
|
||||
PASS = (False, None)
|
||||
CONSUMED = (True, None)
|
||||
|
||||
@@ -34,15 +47,21 @@ class CompositionEngine:
|
||||
self.now = now
|
||||
self.keys = []
|
||||
self.forced_mode = None
|
||||
# True when forced_mode corrects the classifier (learned on commit); False when the
|
||||
# user forced English before typing (Shift tap on an empty buffer).
|
||||
self._forced_is_correction = False
|
||||
# English mode, toggled by a Shift tap when nothing is being typed: keys go straight
|
||||
# to the app (no composition, no underline) until Shift is tapped again.
|
||||
self.english_mode = False
|
||||
self.backtick = False # "`" typed; the next key may be a forced symbol
|
||||
self.fixed = {} # zhuyin segment start -> {(syl_start, syl_end): text}
|
||||
self.candidates = []
|
||||
self.candidate_index = 0
|
||||
self._candidate_segment_start = 0
|
||||
# Composition cursor for choosing candidates: index into _cursor_stops(), None = end.
|
||||
# Composition cursor: a key index into self.keys (None = at the end). ← → move it
|
||||
# between syllables; ↓ opens candidates for the phrase before it; typing there
|
||||
# inserts a syllable (kept in self.inserting until its tone key, then spliced in).
|
||||
self.cursor = None
|
||||
self.inserting = []
|
||||
# Index of a held space (see HOLD_MIN_SCORE), or None.
|
||||
self.held = None
|
||||
self._previous_mode = None
|
||||
self._previous_time = float("-inf")
|
||||
self._cache_key = None
|
||||
@@ -60,12 +79,35 @@ class CompositionEngine:
|
||||
if self.forced_mode:
|
||||
return [(0, len(keys), self.forced_mode)]
|
||||
ctx = self.context
|
||||
cache_key = (tuple(keys), finishing, ctx)
|
||||
held = self.held if self.held is not None and self.held < len(keys) and keys[self.held] == " " else None
|
||||
cache_key = (tuple(keys), finishing, ctx, held)
|
||||
if cache_key != self._cache_key:
|
||||
self._cache_key = cache_key
|
||||
self._cache_value = self.classifier.segment(keys, finishing, ctx)
|
||||
if held is None:
|
||||
self._cache_value = self.classifier.segment(keys, finishing, ctx)
|
||||
else:
|
||||
self._cache_value = self._held_segments(keys, held, finishing)
|
||||
return self._cache_value
|
||||
|
||||
def _held_segments(self, keys, held, finishing):
|
||||
"""The held word is Chinese if the words typed after it start with Chinese."""
|
||||
rest = keys[held + 1:]
|
||||
base = held + 1
|
||||
rest_segs = [(a + base, b + base, m) for a, b, m in self.classifier.segment(rest, finishing, None)]
|
||||
if rest_segs and rest_segs[0][2] == ZHUYIN:
|
||||
return [(0, rest_segs[0][1], ZHUYIN)] + rest_segs[1:]
|
||||
return [(0, base, LITERAL)] + rest_segs
|
||||
|
||||
def _is_ambiguous_word(self, keys):
|
||||
"""Whether keys + space also read as a common Chinese syllable (see HOLD_MIN_SCORE)."""
|
||||
if not all(k.isascii() and k.isalpha() for k in keys):
|
||||
return False
|
||||
p = bopomofo.parse(list(keys) + [" "])
|
||||
if p.pending is not None or len(p.syllables) != 1:
|
||||
return False
|
||||
entries = self.classifier.lexicon.entries([s.syllable for s in p.syllables])
|
||||
return bool(entries) and entries[0][1] >= HOLD_MIN_SCORE
|
||||
|
||||
@property
|
||||
def segments(self):
|
||||
return self._segments(self.keys)
|
||||
@@ -105,42 +147,177 @@ class CompositionEngine:
|
||||
|
||||
# --- cursor ---
|
||||
|
||||
def _cursor_stops(self):
|
||||
"""[(segment index, syllables before the stop)] for every Zhuyin syllable."""
|
||||
segs = self.segments
|
||||
out = []
|
||||
for i, seg in enumerate(segs):
|
||||
def _syllable_ends(self, seg):
|
||||
"""Absolute key index after each syllable of a Zhuyin segment."""
|
||||
return [seg[0] + ps.end for ps in bopomofo.parse(self.keys[seg[0]:seg[1]]).syllables]
|
||||
|
||||
def _cursor_positions(self):
|
||||
"""Key indices the cursor can stop at: the start, after every Zhuyin syllable,
|
||||
and at every segment boundary."""
|
||||
out = {0, len(self.keys)}
|
||||
for seg in self.segments:
|
||||
out.add(seg[1])
|
||||
if seg[2] == ZHUYIN:
|
||||
out.update(self._syllable_ends(seg))
|
||||
return sorted(out)
|
||||
|
||||
def _stop_at(self, pos):
|
||||
"""(segment index, syllables before pos) for the Zhuyin segment pos is in."""
|
||||
segs = self.segments
|
||||
for i, seg in enumerate(segs):
|
||||
if seg[2] == ZHUYIN and seg[0] < pos <= seg[1]:
|
||||
n = len(self._convertible(seg, i < len(segs) - 1)[0])
|
||||
out.extend((i, k) for k in range(1, n + 1))
|
||||
return out
|
||||
local = sum(1 for e in self._syllable_ends(seg) if e <= pos)
|
||||
return (i, min(local, n)) if local else None
|
||||
return None
|
||||
|
||||
def _candidate_stop(self):
|
||||
"""Where ↓ opens candidates: before the cursor, or the last syllable."""
|
||||
if self.cursor is not None:
|
||||
return self._stop_at(self.cursor)
|
||||
segs = self.segments
|
||||
for i in range(len(segs) - 1, -1, -1):
|
||||
if segs[i][2] == ZHUYIN:
|
||||
n = len(self._convertible(segs[i], i < len(segs) - 1)[0])
|
||||
if n:
|
||||
return i, n
|
||||
return None
|
||||
|
||||
def _base_cursor_offset(self):
|
||||
"""Cursor position in the rendered segments (without the syllable being inserted)."""
|
||||
pieces = self._segment_pieces()
|
||||
if self.cursor is None:
|
||||
return sum(len(t) for t, _ in pieces)
|
||||
segs = self.segments
|
||||
for i, seg in enumerate(segs):
|
||||
if not seg[0] < self.cursor <= seg[1]:
|
||||
continue
|
||||
offset = sum(len(t) for t, _ in pieces[:i])
|
||||
if seg[2] == LITERAL:
|
||||
return offset + self.cursor - seg[0]
|
||||
stop = self._stop_at(self.cursor)
|
||||
local = stop[1] if stop else 0
|
||||
for (a, b), text in self._conversion(seg, i < len(segs) - 1)[0]:
|
||||
if b <= local:
|
||||
offset += len(text)
|
||||
elif a < local:
|
||||
offset += min(local - a, len(text))
|
||||
return offset
|
||||
return 0
|
||||
|
||||
@property
|
||||
def cursor_offset(self):
|
||||
"""Cursor position in marked_text (characters)."""
|
||||
pieces = self.marked_pieces
|
||||
stops = self._cursor_stops()
|
||||
if self.cursor is None or not 0 <= self.cursor < len(stops):
|
||||
return sum(len(t) for t, _ in pieces)
|
||||
seg_index, local = stops[self.cursor]
|
||||
offset = sum(len(t) for t, _ in pieces[:seg_index])
|
||||
segs = self.segments
|
||||
for (a, b), text in self._conversion(segs[seg_index], seg_index < len(segs) - 1)[0]:
|
||||
if b <= local:
|
||||
offset += len(text)
|
||||
elif a < local:
|
||||
offset += min(local - a, len(text))
|
||||
return offset
|
||||
return self._base_cursor_offset() + len(self._inserting_text())
|
||||
|
||||
def _move_cursor(self, delta):
|
||||
stops = self._cursor_stops()
|
||||
if not stops:
|
||||
return
|
||||
nxt = (len(stops) - 1 if self.cursor is None else self.cursor) + delta
|
||||
self.cursor = None if nxt >= len(stops) - 1 else max(0, nxt)
|
||||
self.inserting = []
|
||||
stops = self._cursor_positions()
|
||||
here = len(self.keys) if self.cursor is None else self.cursor
|
||||
if delta < 0:
|
||||
before = [p for p in stops if p < here]
|
||||
nxt = before[-1] if before else here
|
||||
else:
|
||||
after = [p for p in stops if p > here]
|
||||
nxt = after[0] if after else here
|
||||
self.cursor = None if nxt >= len(self.keys) else nxt
|
||||
|
||||
def _inserting_text(self):
|
||||
if not self.inserting:
|
||||
return ""
|
||||
return "".join(ps.syllable.reading for ps in bopomofo.parse(self.inserting).syllables)
|
||||
|
||||
# --- editing in the middle ---
|
||||
|
||||
def _trim_fixed(self, pos):
|
||||
"""Keeps picked phrases that end at or before key index pos (they don't move)."""
|
||||
kept = {}
|
||||
for seg in self.segments:
|
||||
picks = self.fixed.get(seg[0])
|
||||
if not picks or seg[2] != ZHUYIN:
|
||||
continue
|
||||
if seg[1] <= pos:
|
||||
kept[seg[0]] = picks
|
||||
elif seg[0] < pos:
|
||||
ends = self._syllable_ends(seg)
|
||||
here = {r: t for r, t in picks.items() if r[1] <= len(ends) and ends[r[1] - 1] <= pos}
|
||||
if here:
|
||||
kept[seg[0]] = here
|
||||
self.fixed = kept
|
||||
|
||||
def _splice(self, new_keys):
|
||||
"""Inserts keys at the cursor and moves the cursor after them."""
|
||||
pos = self.cursor
|
||||
self._trim_fixed(pos)
|
||||
self.keys[pos:pos] = new_keys
|
||||
if self.held is not None and self.held >= pos:
|
||||
self.held += len(new_keys)
|
||||
self.cursor = pos + len(new_keys)
|
||||
if self.cursor >= len(self.keys):
|
||||
self.cursor = None
|
||||
|
||||
def _delete_before_cursor(self):
|
||||
"""Backspace in the middle: removes the syllable (or literal key) before the cursor."""
|
||||
pos = self.cursor
|
||||
start = pos - 1
|
||||
for seg in self.segments:
|
||||
if seg[2] == ZHUYIN and seg[0] < pos <= seg[1]:
|
||||
for ps in bopomofo.parse(self.keys[seg[0]:seg[1]]).syllables:
|
||||
if seg[0] + ps.end == pos:
|
||||
start = seg[0] + ps.start
|
||||
self._trim_fixed(start)
|
||||
del self.keys[start:pos]
|
||||
if self.held is not None:
|
||||
if start <= self.held < pos:
|
||||
self.held = None
|
||||
elif self.held >= pos:
|
||||
self.held -= pos - start
|
||||
self.cursor = start
|
||||
if not self.keys:
|
||||
self.reset()
|
||||
|
||||
def _insert_key(self, c):
|
||||
"""Typing with the cursor in the middle: compose one syllable, splice it in on its tone."""
|
||||
if c == " ":
|
||||
if not self.inserting:
|
||||
return True, self._commit()
|
||||
self._splice(self.inserting + [" "])
|
||||
self.inserting = []
|
||||
return CONSUMED
|
||||
self.inserting.append(c)
|
||||
p = bopomofo.parse(self.inserting)
|
||||
if p.stray_tones or p.foreign_keys:
|
||||
self.inserting.pop() # a tone with nothing to attach to
|
||||
return CONSUMED
|
||||
if p.syllables and p.syllables[-1].toned and len(p.syllables) == 1:
|
||||
self._splice(self.inserting)
|
||||
self.inserting = []
|
||||
elif len(p.syllables) > 1:
|
||||
# A new syllable started without a tone: splice the first one as it is.
|
||||
first = p.syllables[0].end
|
||||
self._splice(self.inserting[:first])
|
||||
self.inserting = self.inserting[first:]
|
||||
return CONSUMED
|
||||
|
||||
@property
|
||||
def marked_pieces(self):
|
||||
pieces = self._segment_pieces()
|
||||
if self.inserting:
|
||||
# Show the syllable being inserted at the cursor.
|
||||
at, out, done = self._base_cursor_offset(), [], False
|
||||
for text, mode in pieces:
|
||||
if not done and at <= len(text):
|
||||
out += [(text[:at], mode), (self._inserting_text(), ZHUYIN), (text[at:], mode)]
|
||||
done = True
|
||||
else:
|
||||
out.append((text, mode))
|
||||
at -= len(text)
|
||||
pieces = [(t, m) for t, m in out if t]
|
||||
if self.backtick:
|
||||
pieces.append((BACKTICK, LITERAL))
|
||||
return pieces
|
||||
|
||||
def _segment_pieces(self):
|
||||
segs = self.segments
|
||||
return [(self._render(s, i < len(segs) - 1), s[2]) for i, s in enumerate(segs)]
|
||||
|
||||
@@ -150,28 +327,20 @@ class CompositionEngine:
|
||||
|
||||
# --- key handling ---
|
||||
|
||||
@property
|
||||
def english_armed(self):
|
||||
"""True while the user has forced English for the next word (Shift tap). In this
|
||||
mode keys go straight to the app with no composition (no underline)."""
|
||||
return not self.keys and self.forced_mode == LITERAL and not self._forced_is_correction
|
||||
|
||||
def _end_direct_english(self):
|
||||
if self.english_armed:
|
||||
self.forced_mode = None
|
||||
self._previous_mode = LITERAL
|
||||
self._previous_time = self.now()
|
||||
|
||||
def _flip(self):
|
||||
"""Forces the whole buffer into the other mode (a correction that is learned)."""
|
||||
self.forced_mode = LITERAL if self.mode == ZHUYIN else ZHUYIN
|
||||
self._forced_is_correction = True
|
||||
|
||||
def wants(self, key):
|
||||
"""Whether handle(key) would consume the key, without changing state."""
|
||||
if self.candidates or key == SHIFT_TAP or self.english_armed:
|
||||
if self.candidates or key == SHIFT_TAP or self.backtick or self.inserting:
|
||||
return True
|
||||
if self.english_mode and not self.keys:
|
||||
return False # English mode: everything goes to the app as typed
|
||||
if key in SPECIAL_KEYS:
|
||||
return bool(self.keys)
|
||||
if key == BACKTICK:
|
||||
return True
|
||||
if self.keys:
|
||||
return True
|
||||
if key == " ":
|
||||
@@ -183,48 +352,60 @@ class CompositionEngine:
|
||||
def handle(self, key):
|
||||
if self.candidates:
|
||||
return self._handle_candidate_key(key)
|
||||
if self.backtick:
|
||||
return self._after_backtick(key)
|
||||
if self.inserting and key in (BACKSPACE, ESCAPE):
|
||||
if key == BACKSPACE:
|
||||
self.inserting.pop()
|
||||
else:
|
||||
self.inserting = []
|
||||
return CONSUMED
|
||||
if key == BACKSPACE:
|
||||
if not self.keys:
|
||||
return PASS
|
||||
self.cursor = None
|
||||
if self.cursor is not None:
|
||||
if self.cursor > 0:
|
||||
self._delete_before_cursor()
|
||||
return CONSUMED
|
||||
self.keys.pop()
|
||||
if self.held is not None and len(self.keys) <= self.held:
|
||||
self.held = None
|
||||
self.fixed = {k: v for k, v in self.fixed.items() if k < len(self.keys)}
|
||||
if not self.keys:
|
||||
# A Shift-tap "next word is English" stays armed; a correction is dropped.
|
||||
if self._forced_is_correction:
|
||||
self.forced_mode = None
|
||||
self._forced_is_correction = False
|
||||
self.forced_mode = None
|
||||
self.fixed = {}
|
||||
return CONSUMED
|
||||
if key == ENTER:
|
||||
if not self.keys:
|
||||
self._end_direct_english()
|
||||
return PASS
|
||||
if self.inserting:
|
||||
self._splice(self.inserting)
|
||||
self.inserting = []
|
||||
return True, self._commit()
|
||||
if key == ESCAPE:
|
||||
if not self.keys:
|
||||
self._end_direct_english()
|
||||
return PASS
|
||||
self.reset()
|
||||
return CONSUMED
|
||||
if key == TAB:
|
||||
if not self.keys:
|
||||
self._end_direct_english()
|
||||
return PASS
|
||||
self._flip()
|
||||
return CONSUMED
|
||||
if key == SHIFT_TAP:
|
||||
if not self.keys:
|
||||
# Next word is English, typed as-is. Tap again to cancel.
|
||||
self.forced_mode = None if self.forced_mode == LITERAL else LITERAL
|
||||
self._forced_is_correction = False
|
||||
# Toggle English mode; it stays on until the next Shift tap.
|
||||
self.english_mode = not self.english_mode
|
||||
self._previous_mode = LITERAL if self.english_mode else None
|
||||
self._previous_time = self.now()
|
||||
else:
|
||||
self._flip()
|
||||
return CONSUMED
|
||||
if key == DOWN:
|
||||
if not self.keys:
|
||||
return PASS
|
||||
self._open_candidates()
|
||||
if not self.inserting:
|
||||
self._open_candidates()
|
||||
return CONSUMED
|
||||
if key in (LEFT, RIGHT):
|
||||
if not self.keys:
|
||||
@@ -235,6 +416,19 @@ class CompositionEngine:
|
||||
return CONSUMED if self.keys else PASS
|
||||
return self._handle_character(key)
|
||||
|
||||
def _after_backtick(self, key):
|
||||
self.backtick = False
|
||||
if key in (ESCAPE, BACKSPACE):
|
||||
return CONSUMED
|
||||
committed = self._commit() if self.keys else ""
|
||||
if key in SYMBOLS:
|
||||
return True, committed + SYMBOLS[key]
|
||||
# Not a symbol: the backtick was meant literally; then handle the key as usual.
|
||||
handled, more = self.handle(key)
|
||||
if not handled and len(key) == 1:
|
||||
handled, more = True, key
|
||||
return handled, committed + BACKTICK + (more or "")
|
||||
|
||||
def _last_parse(self, segs):
|
||||
if not segs or segs[-1][2] != ZHUYIN:
|
||||
return None
|
||||
@@ -242,21 +436,27 @@ class CompositionEngine:
|
||||
return bopomofo.parse(self.keys[start:end])
|
||||
|
||||
def _handle_character(self, c):
|
||||
self.cursor = None # typing always continues at the end
|
||||
if self.english_armed:
|
||||
# Forced English: insert directly. Space ends the word; other symbols end it too.
|
||||
if c == " ":
|
||||
self._end_direct_english()
|
||||
return True, " "
|
||||
if c.isascii() and (c.isalnum() or c in "'.,-/;"):
|
||||
return True, c
|
||||
self._end_direct_english()
|
||||
return PASS
|
||||
if self.cursor is not None and self.keys and (c == " " or bopomofo.is_zhuyin_key(c)):
|
||||
return self._insert_key(c)
|
||||
if self.inserting:
|
||||
self._splice(self.inserting)
|
||||
self.inserting = []
|
||||
self.cursor = None # anything else continues at the end
|
||||
if self.english_mode and not self.keys:
|
||||
return PASS # English mode: every key goes to the app as typed
|
||||
if c == BACKTICK:
|
||||
self.backtick = True
|
||||
return CONSUMED
|
||||
if c == " ":
|
||||
if not self.keys:
|
||||
return PASS
|
||||
segs = self._segments(self.keys, finishing=True)
|
||||
if segs[-1][2] == LITERAL:
|
||||
if (self.held is None and len(segs) == 1 and not self.forced_mode
|
||||
and self._is_ambiguous_word(self.keys)):
|
||||
self.held = len(self.keys)
|
||||
self.keys.append(" ")
|
||||
return CONSUMED
|
||||
return True, self._commit(segs) + " "
|
||||
p = self._last_parse(segs)
|
||||
if p is not None and p.pending is not None:
|
||||
@@ -285,10 +485,10 @@ class CompositionEngine:
|
||||
|
||||
def _open_candidates(self):
|
||||
"""Candidates for the phrase ending at the cursor (default: the last syllable)."""
|
||||
stops = self._cursor_stops()
|
||||
if not stops:
|
||||
stop = self._candidate_stop()
|
||||
if stop is None:
|
||||
return
|
||||
seg_index, local = stops[min(len(stops) - 1 if self.cursor is None else self.cursor, len(stops) - 1)]
|
||||
seg_index, local = stop
|
||||
segs = self.segments
|
||||
seg = segs[seg_index]
|
||||
syl = self._convertible(seg, seg_index < len(segs) - 1)[0]
|
||||
@@ -329,7 +529,7 @@ class CompositionEngine:
|
||||
def _commit(self, segs=None):
|
||||
segs = segs if segs is not None else self.segments
|
||||
text = "".join(self._render(s, True) for s in segs)
|
||||
if self.forced_mode and self._forced_is_correction:
|
||||
if self.forced_mode:
|
||||
self.classifier.learned.record("".join(self.keys), self.forced_mode)
|
||||
self._previous_mode = segs[-1][2] if segs else None
|
||||
self._previous_time = self.now()
|
||||
@@ -341,8 +541,10 @@ class CompositionEngine:
|
||||
|
||||
def reset(self):
|
||||
self.keys = []
|
||||
self.inserting = []
|
||||
self.held = None
|
||||
self.backtick = False
|
||||
self.cursor = None
|
||||
self.forced_mode = None
|
||||
self._forced_is_correction = False
|
||||
self.fixed = {}
|
||||
self.candidates = []
|
||||
|
||||
@@ -41,6 +41,9 @@ DEFAULT_WEIGHTS = {
|
||||
"lit.others": -1.5,
|
||||
# Segmentation
|
||||
"seg.switch": -3.0,
|
||||
# Context: the first segment has the same mode as the previously committed text
|
||||
"ctx.zh": 1.5,
|
||||
"ctx.lit": 1.5,
|
||||
}
|
||||
|
||||
|
||||
@@ -52,7 +55,7 @@ def _add(f, k, v=1.0):
|
||||
f[k] = f.get(k, 0.0) + v
|
||||
|
||||
|
||||
def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=False):
|
||||
def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=False, previous=None):
|
||||
"""Features of `keys` read as Zhuyin, or None if impossible."""
|
||||
parse = bopomofo.parse(keys)
|
||||
if not is_last and parse.pending is not None:
|
||||
@@ -99,10 +102,12 @@ def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=Fa
|
||||
_add(f, "zh.unconfirmed")
|
||||
if after_other:
|
||||
_add(f, "seg.switch")
|
||||
elif previous == "zhuyin":
|
||||
_add(f, "ctx.zh")
|
||||
return f
|
||||
|
||||
|
||||
def literal_features(keys, english, is_last=True, after_other=False, finishing=False):
|
||||
def literal_features(keys, english, is_last=True, after_other=False, finishing=False, previous=None):
|
||||
"""Features of `keys` output as typed, or None if impossible."""
|
||||
if not keys or " " in keys:
|
||||
return None
|
||||
@@ -110,6 +115,8 @@ def literal_features(keys, english, is_last=True, after_other=False, finishing=F
|
||||
f = {"lit.bias": 1.0}
|
||||
if after_other:
|
||||
_add(f, "seg.switch")
|
||||
elif previous == "literal":
|
||||
_add(f, "ctx.lit")
|
||||
if any(c.isupper() for c in keys):
|
||||
_add(f, "lit.upper")
|
||||
letters = "".join(c for c in keys if c.isalpha())
|
||||
|
||||
@@ -16,7 +16,7 @@ class Lexicon:
|
||||
|
||||
def __init__(self):
|
||||
self.exact = {}
|
||||
self.toneless = {}
|
||||
self._toneless = None # built on first use (only untoned syllables need it)
|
||||
self.max_phrase_length = 1
|
||||
|
||||
@classmethod
|
||||
@@ -42,9 +42,10 @@ class Lexicon:
|
||||
except ValueError:
|
||||
score = -6.0
|
||||
self.add(reading, phrase, score)
|
||||
for table in (self.exact, self.toneless):
|
||||
for v in table.values():
|
||||
for v in self.exact.values():
|
||||
if len(v) > 1:
|
||||
v.sort(key=lambda e: -e[1])
|
||||
self._toneless = None
|
||||
|
||||
def add(self, reading, phrase, score):
|
||||
syllables = [s[1:] + "˙" if s.startswith("˙") else s for s in reading.split("-")]
|
||||
@@ -53,15 +54,20 @@ class Lexicon:
|
||||
entries = self.exact.setdefault(reading, [])
|
||||
if not any(p == phrase for p, _ in entries):
|
||||
entries.append((phrase, score))
|
||||
key = _strip_tones(reading)
|
||||
tl = self.toneless.setdefault(key, [])
|
||||
for i, (p, s) in enumerate(tl):
|
||||
if p == phrase:
|
||||
if score > s:
|
||||
tl[i] = (p, score)
|
||||
break
|
||||
else:
|
||||
tl.append((phrase, score))
|
||||
self._toneless = None
|
||||
|
||||
@property
|
||||
def toneless(self):
|
||||
"""Index ignoring tones: toneless reading -> [(phrase, best score)], best first."""
|
||||
if self._toneless is None:
|
||||
best = {}
|
||||
for reading, entries in self.exact.items():
|
||||
bucket = best.setdefault(_strip_tones(reading), {})
|
||||
for phrase, score in entries:
|
||||
if score > bucket.get(phrase, float("-inf")):
|
||||
bucket[phrase] = score
|
||||
self._toneless = {k: sorted(v.items(), key=lambda e: -e[1]) for k, v in best.items()}
|
||||
return self._toneless
|
||||
|
||||
def entries(self, syllables):
|
||||
if all(s.tone is not None for s in syllables):
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "自動注音 AutoZhuyin",
|
||||
"version": "0.5.1",
|
||||
"version": "0.9.0",
|
||||
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
||||
"locale": "zh-Hant-TW",
|
||||
"fallbackLocale": "zh-TW",
|
||||
|
||||
@@ -26,14 +26,18 @@
|
||||
空白 注音:一聲;音節打完時送出中文。英文:送出單字加空白
|
||||
Enter 送出目前組字
|
||||
Tab 判斷錯了?按 Tab 在中/英之間翻轉,輸入法會記住這次修正
|
||||
← → 在組字區移動游標,選要改哪個字
|
||||
← → 在組字區移動游標,選要改哪個字;游標在中間時直接打注音
|
||||
會插入新字,Backspace 刪掉游標前的字
|
||||
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字
|
||||
Esc 取消組字
|
||||
單按 Shift 下一個字強制英文(保持小寫、直接送出、沒有底線),
|
||||
按空白或 Enter 後恢復自動判斷;
|
||||
打到一半時按,同 Tab 整段翻轉。會出現「英文」小提示
|
||||
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線),
|
||||
再單按一次回到中英自動判斷,會出現「英文」/「中英自動」小提示;
|
||||
打到一半時按,同 Tab 整段翻轉
|
||||
Shift+字母 打大寫字母(也會強制英文)
|
||||
< > ? ! : 中文語境下自動變成全形 ,。?!:
|
||||
` + 標點 強制全形標點,例如 `< 或 `, = , `. = 。 `/ = ? `; = ;
|
||||
`' = 、 `[ `] = 「」 `( `) = () `! = !
|
||||
` 後面接其他鍵就是普通的反引號
|
||||
|
||||
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
|
||||
|
||||
|
||||
+93
-32
@@ -42,7 +42,7 @@ def make_engine():
|
||||
def type_keys(s, e):
|
||||
out = ""
|
||||
for c in s:
|
||||
key = {"\n": ENTER, "\t": TAB, "⇧": SHIFT_TAP, "←": LEFT, "→": RIGHT, "↓": DOWN}.get(c, c)
|
||||
key = {"\n": ENTER, "\t": TAB, "⇧": SHIFT_TAP, "←": LEFT, "→": RIGHT, "↓": DOWN, "\b": BACKSPACE}.get(c, c)
|
||||
handled, commit = e.handle(key)
|
||||
if commit is not None:
|
||||
out += commit
|
||||
@@ -194,11 +194,15 @@ class MixedSegmentTests(unittest.TestCase):
|
||||
|
||||
|
||||
class ShiftTapTests(unittest.TestCase):
|
||||
def test_forces_next_word_english_lowercase(self):
|
||||
def test_toggles_english_mode(self):
|
||||
e = make_engine()
|
||||
self.assertEqual(type_keys("⇧ru ", e), "ru ")
|
||||
self.assertFalse(e.english_armed)
|
||||
self.assertEqual(type_keys("su3cl3\n", e), "你好")
|
||||
self.assertEqual(type_keys("⇧ru su ", e), "ru su ")
|
||||
self.assertTrue(e.english_mode)
|
||||
self.assertEqual(e.marked_text, "")
|
||||
self.assertEqual(e.handle(ENTER), (False, None))
|
||||
self.assertTrue(e.english_mode)
|
||||
self.assertEqual(type_keys("⇧su3cl3\n", e), "你好")
|
||||
self.assertFalse(e.english_mode)
|
||||
|
||||
def test_tap_twice_cancels(self):
|
||||
self.assertEqual(type_keys("⇧⇧su3cl3\n", make_engine()), "你好")
|
||||
@@ -208,20 +212,20 @@ class ShiftTapTests(unittest.TestCase):
|
||||
type_keys("so", e)
|
||||
e.handle(SHIFT_TAP)
|
||||
self.assertEqual(e.mode, ZHUYIN)
|
||||
self.assertFalse(e.english_mode)
|
||||
type_keys("\n", e)
|
||||
self.assertEqual(e.classifier.learned.weights["so"], 1)
|
||||
|
||||
def test_forcing_before_typing_not_learned(self):
|
||||
def test_english_mode_not_learned(self):
|
||||
e = make_engine()
|
||||
type_keys("⇧hello ", e)
|
||||
self.assertNotIn("hello", e.classifier.learned.weights)
|
||||
self.assertEqual(e.classifier.learned.weights, {})
|
||||
|
||||
def test_arming_survives_backspace(self):
|
||||
def test_backspace_goes_to_app(self):
|
||||
e = make_engine()
|
||||
type_keys("⇧r", e)
|
||||
e.handle(BACKSPACE)
|
||||
self.assertTrue(e.english_armed)
|
||||
self.assertEqual(type_keys("ru ", e), "ru ")
|
||||
e.handle(SHIFT_TAP)
|
||||
self.assertEqual(e.handle(BACKSPACE), (False, None))
|
||||
self.assertTrue(e.english_mode)
|
||||
|
||||
def test_detector(self):
|
||||
d = ShiftTapDetector()
|
||||
@@ -242,7 +246,7 @@ class CursorTests(unittest.TestCase):
|
||||
type_keys("←", e)
|
||||
self.assertEqual(e.cursor_offset, 3)
|
||||
type_keys("←←←←", e)
|
||||
self.assertEqual(e.cursor_offset, 1)
|
||||
self.assertEqual(e.cursor_offset, 0)
|
||||
type_keys("→→→→", e)
|
||||
self.assertEqual(e.cursor_offset, 4)
|
||||
|
||||
@@ -254,12 +258,45 @@ class CursorTests(unittest.TestCase):
|
||||
e.select_candidate([t for _, t in e.candidates].index("添"))
|
||||
self.assertEqual(type_keys("\n", e), "今添天氣")
|
||||
|
||||
def test_typing_returns_cursor_to_end(self):
|
||||
def test_english_returns_cursor_to_end(self):
|
||||
e = make_engine()
|
||||
type_keys("rup wu0 ←", e)
|
||||
self.assertEqual(e.cursor_offset, 1)
|
||||
type_keys("wu0 ", e)
|
||||
self.assertEqual(e.cursor_offset, len(e.marked_text))
|
||||
type_keys("?", e)
|
||||
self.assertEqual(e.marked_text, "")
|
||||
|
||||
|
||||
class InsertInMiddleTests(unittest.TestCase):
|
||||
def test_insert_syllable(self):
|
||||
e = make_engine()
|
||||
type_keys("rup fu4←wu", e)
|
||||
self.assertEqual(e.marked_text, "今ㄊㄧ氣")
|
||||
self.assertEqual(e.cursor_offset, 3)
|
||||
type_keys("0 ", e)
|
||||
self.assertEqual(e.marked_text, "今天氣")
|
||||
self.assertEqual(e.cursor_offset, 2)
|
||||
self.assertEqual(type_keys("\n", e), "今天氣")
|
||||
|
||||
def test_insert_at_start(self):
|
||||
self.assertEqual(type_keys("su3cl3←←ji3\n", make_engine()), "我你好")
|
||||
|
||||
def test_backspace_deletes_character_before_cursor(self):
|
||||
e = make_engine()
|
||||
type_keys("rup wu0 wu0 fu4←←\b", e)
|
||||
self.assertEqual(e.marked_text, "今天氣")
|
||||
self.assertEqual(e.cursor_offset, 1)
|
||||
|
||||
def test_backspace_inside_inserted_syllable(self):
|
||||
e = make_engine()
|
||||
type_keys("rup fu4←wu\b", e)
|
||||
self.assertEqual(e.marked_text, "今ㄊ氣")
|
||||
|
||||
def test_keeps_picked_phrase_before_cursor(self):
|
||||
e = make_engine()
|
||||
type_keys("rup wu0 ↓", e)
|
||||
e.select_candidate([t for _, t in e.candidates].index("添"))
|
||||
type_keys("fu4←wu0 ", e) # insert 天 between 添 and 氣
|
||||
self.assertEqual(e.marked_text[:3], "今添天")
|
||||
|
||||
def test_candidates_for_chinese_before_english(self):
|
||||
e = make_engine()
|
||||
@@ -267,24 +304,48 @@ class CursorTests(unittest.TestCase):
|
||||
self.assertTrue(e.showing_candidates)
|
||||
|
||||
|
||||
class DirectEnglishTests(unittest.TestCase):
|
||||
def test_forced_english_has_no_composition(self):
|
||||
e = make_engine()
|
||||
e.handle(SHIFT_TAP)
|
||||
self.assertEqual(e.handle("r"), (True, "r"))
|
||||
self.assertEqual(e.marked_text, "")
|
||||
self.assertEqual(e.handle("'"), (True, "'"))
|
||||
self.assertEqual(e.handle(BACKSPACE), (False, None))
|
||||
self.assertTrue(e.english_armed)
|
||||
self.assertEqual(e.handle(" "), (True, " "))
|
||||
self.assertFalse(e.english_armed)
|
||||
class FirstToneAfterEnglishTests(unittest.TestCase):
|
||||
def test_first_tone_after_english(self):
|
||||
for keys in ["ru", "t", "gj", "vup", "tj", "el"]:
|
||||
e = make_engine()
|
||||
self.assertEqual(type_keys("hello " + keys + " ", e), "hello ", keys)
|
||||
self.assertEqual(len(e.marked_text), 1, keys)
|
||||
|
||||
def test_enter_ends_forced_english(self):
|
||||
|
||||
class HeldWordTests(unittest.TestCase):
|
||||
def test_next_word_decides(self):
|
||||
cases = {"up jo4 ": "因為", "up the ": "up the ", "pick up the ": "pick up the ",
|
||||
"up\n": "up", "i love ": "i love "}
|
||||
for keys, want in cases.items():
|
||||
self.assertEqual(type_keys(keys, make_engine()), want, keys)
|
||||
|
||||
def test_held_word_stays_composing(self):
|
||||
e = make_engine()
|
||||
e.handle(SHIFT_TAP)
|
||||
e.handle("a")
|
||||
self.assertEqual(e.handle(ENTER), (False, None))
|
||||
self.assertFalse(e.english_armed)
|
||||
self.assertEqual(type_keys("up ", e), "")
|
||||
self.assertEqual(e.marked_text, "up ")
|
||||
e.handle(BACKSPACE)
|
||||
self.assertEqual(e.marked_text, "up")
|
||||
self.assertEqual(type_keys(" the ", e), "up the ")
|
||||
|
||||
|
||||
class BacktickSymbolTests(unittest.TestCase):
|
||||
def test_forced_symbols(self):
|
||||
cases = {"`<": ",", "`.": "。", "`?": "?", "`;": ";", "`[": "「",
|
||||
"hello`,": "hello,", "su3cl3`.": "你好。", "```\n": "```"}
|
||||
for keys, want in cases.items():
|
||||
self.assertEqual(type_keys(keys, make_engine()), want, keys)
|
||||
|
||||
def test_plain_backtick(self):
|
||||
e = make_engine()
|
||||
self.assertEqual(type_keys("`", e), "")
|
||||
self.assertEqual(e.marked_text, "`")
|
||||
self.assertEqual(type_keys("ls` ", e), "`ls` ")
|
||||
|
||||
def test_escape_cancels(self):
|
||||
e = make_engine()
|
||||
type_keys("su3`", e)
|
||||
e.handle(BACKSPACE)
|
||||
self.assertEqual(e.marked_text, "你")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -105,6 +105,10 @@ class WrapperTests(unittest.TestCase):
|
||||
def test_mixed_sentence(self):
|
||||
self.assertEqual(type_text(self.svc, "ji3 hello world su3cl3 "), "我hello world 你好")
|
||||
|
||||
def test_backtick_symbol(self):
|
||||
self.assertEqual(type_text(self.svc, "`<"), ",")
|
||||
self.assertEqual(type_text(self.svc, "ji3`."), "我。")
|
||||
|
||||
def test_composition_shown(self):
|
||||
type_text(self.svc, "su3c")
|
||||
self.assertEqual(self.svc.reply["compositionString"], "你ㄏ")
|
||||
@@ -144,14 +148,15 @@ class ShiftTapWrapperTests(unittest.TestCase):
|
||||
def test_tap_forces_english_lowercase(self):
|
||||
reply = tap_shift(self.svc)
|
||||
self.assertEqual(reply.get("showMessage"), "英文")
|
||||
self.assertEqual(type_text(self.svc, "ru "), "ru ")
|
||||
self.assertEqual(type_text(self.svc, "ru su "), "ru su ")
|
||||
self.assertEqual(tap_shift(self.svc).get("showMessage"), "中英自動")
|
||||
|
||||
def test_shift_with_letter_is_not_a_tap(self):
|
||||
self.svc.filterKeyDown(FakeKeyEvent("⇧", shift=True))
|
||||
press(self.svc, "H", shift=True)
|
||||
up = FakeKeyEvent("⇧")
|
||||
self.assertFalse(self.svc.filterKeyUp(up))
|
||||
self.assertFalse(self.svc.engine.english_armed)
|
||||
self.assertFalse(self.svc.engine.english_mode)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user