Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cfd095b3b0 | ||
|
|
f323a29552 | ||
|
|
aa00b793ce | ||
|
|
b3b7589379 | ||
|
|
c440cd7177 | ||
|
|
6443f771a0 | ||
|
|
0155ddd16e | ||
|
|
c8b3c4ae6f |
@@ -12,12 +12,12 @@
|
||||
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
|
||||
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
|
||||
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
|
||||
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷
|
||||
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷;英文模式時輸入法圖示會從「注」變成「A」
|
||||
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
|
||||
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
|
||||
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
|
||||
- **記住選字**:從候選字選過的字詞會被記住,下次打同樣的讀音會優先出現(選越多次越優先)
|
||||
- **AI 選字(macOS)**:用中研院 CKIP 的繁中小型語言模型看前後文挑同音字(他跑「得」很快、請你「再」說一次),完全在本機運算;可調整或關閉,系統忙碌時自動暫停
|
||||
- **AI 選字**:用中研院 CKIP 的繁中小型語言模型看前後文挑同音字(他跑「得」很快、請你「再」說一次),完全在本機運算;可調整或關閉,系統忙碌時自動暫停
|
||||
- **中間補字、刪字**:游標移到前面後直接打注音,會插在游標處;Backspace 刪掉游標前的那個字
|
||||
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`;任何時候按 `` ` `` 再按標點鍵,一定得到全形標點(例如 `` `< `` = `,`)
|
||||
- **完整詞庫**:約 17 萬筆,來自小麥注音 McBopomofo(MIT 授權)
|
||||
@@ -34,6 +34,8 @@
|
||||
2. 系統設定 → 鍵盤 → 文字輸入 → 編輯… → + → 繁體中文 → **自動注音**。
|
||||
3. 清單裡沒出現的話,登出再登入一次。
|
||||
|
||||
從 0.12 以前的版本更新:輸入法現在分成「中英自動」和「英文」兩個模式(各有圖示),安裝時會自動加入。如果輸入法清單裡還留著舊的「自動注音」項目,把它移除即可。
|
||||
|
||||
PKG 會裝到 `/Library/Input Methods/`,系統上所有使用者都能用。
|
||||
|
||||
### Windows
|
||||
@@ -67,7 +69,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
| ← → | 在組字區移動游標(最左停在第一個中文字後面)。游標在中間時:直接打注音會**插入**新字,Backspace 刪掉游標前的字,↓ 選游標前的字 |
|
||||
| ↓ | 開啟游標前那個字或詞的候選字(游標預設在最後),數字鍵 1–9 選字,Esc 關閉 |
|
||||
| Esc | 取消組字 |
|
||||
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷。打到一半時:同 Tab,整段翻轉 |
|
||||
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷,圖示會變成「A」(Mac 的選單列、Windows 的工作列、Linux 的 IBus 狀態)。Mac 也可以直接在選單列選「英文」模式。打到一半時:同 Tab,整段翻轉 |
|
||||
| Shift + 字母 | 打大寫字母(也會強制英文) |
|
||||
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
|
||||
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
|
||||
@@ -97,7 +99,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
|
||||
轉換用 Viterbi 演算法,在詞庫中找出整串音節機率最高的詞組組合。
|
||||
|
||||
### AI 選字(macOS)
|
||||
### AI 選字
|
||||
|
||||
詞庫只知道每個詞本身有多常見,不看前後文。AI 選字讓語言模型當裁判:
|
||||
|
||||
@@ -105,7 +107,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
2. 模型 [CKIP GPT2-tiny](https://huggingface.co/ckiplab/gpt2-tiny-chinese)(中研院詞庫小組,繁體中文,約 1200 萬參數,22 MB)計算每個組合出現的機率,並參考剛送出的前文。
|
||||
3. 分數 = 詞庫分數 + 0.6 × 模型分數,取最高的。選過的候選字仍然固定不變。
|
||||
|
||||
輸入法選單可以調整:
|
||||
輸入法選單可以調整(Mac:選單列的輸入法選單;Windows:工作列輸入法圖示按右鍵或語言列的設定按鈕;Linux:IBus 選單的「設定」):
|
||||
|
||||
| 模式 | 比較幾個組合 | 每次運算上限 |
|
||||
|---|---|---|
|
||||
@@ -116,7 +118,13 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
|
||||
不會拖慢打字的機制:
|
||||
- 每次運算超過上限就直接放棄,改用詞庫結果;連續 3 次超時就暫停一分鐘。
|
||||
- 「系統忙碌時自動暫停 AI」(預設開啟):Mac 過熱,或 CPU 負載(每核心 1 分鐘平均)超過 0.9 時暫停;低耗電模式下降到省電模式。選單最下面會顯示目前狀態。
|
||||
- 「系統忙碌時自動暫停 AI」(預設開啟),選單裡會顯示目前狀態:
|
||||
|
||||
| | 暫停 AI | 降到省電模式 |
|
||||
|---|---|---|
|
||||
| macOS | 過熱,或 CPU 負載(每核心 1 分鐘平均)超過 0.9 | 低耗電模式 |
|
||||
| Windows | CPU 使用率超過 90% | 省電模式(電池保護) |
|
||||
| Linux | CPU 負載超過 0.9,或溫度達 95 °C | 低耗電設定檔,或用電池且電量 ≤ 20% |
|
||||
- 模型在背景載入,載入完成前照常用詞庫。
|
||||
|
||||
實測(`training/lm/eval_rerank.py`,標準模式):
|
||||
@@ -126,7 +134,12 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
| 日常對話 60 句 | 字正確率 96.6%,整句全對 73% | 97.5%,80% |
|
||||
| 維基百科 714 句 | 92.9%,31% | 94.7%,44% |
|
||||
|
||||
錯字大約少了 25–30%。Windows、Linux 版目前還沒有 AI 選字。
|
||||
錯字大約少了 25–30%。
|
||||
|
||||
Windows、Linux 版用 numpy 執行同一個模型(`windows/autozhuyin/core/lm.py`),所以時間上限放寬為省電 40、標準 80、高品質 200 毫秒,超時的那次結果照用但不快取,連續 3 次超時暫停一分鐘。
|
||||
- Windows:安裝包附上 PIME 兩種 Python(1.3 版的 3.8、2.x 版的 3.15,都是 32 位元)用的 numpy,安裝檔約 40 MB。
|
||||
- Linux:系統沒有 numpy 時,`install.sh` 會從 PyPI 下載符合的版本到 `~/.local/share/autozhuyin/vendor/`。
|
||||
- 設定存在 `%APPDATA%\PIME\autozhuyin\settings.json`、`~/.config/autozhuyin/settings.json`。
|
||||
|
||||
## 個人資料
|
||||
|
||||
@@ -311,8 +324,7 @@ THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝
|
||||
- [x] 用資料訓練判斷權重(合成資料)
|
||||
- [ ] 加入真實打字資料一起訓練
|
||||
- [ ] 已送出的英文單字若判斷錯誤,可以「事後翻轉」
|
||||
- [x] AI 選字:語言模型參考前後文挑同音字(macOS)
|
||||
- [ ] AI 選字移植到 Windows、Linux
|
||||
- [x] AI 選字:語言模型參考前後文挑同音字(macOS、Windows、Linux)
|
||||
- [x] 使用者詞頻學習(選過的候選字優先)
|
||||
- [ ] 偏好設定視窗
|
||||
|
||||
@@ -321,6 +333,7 @@ THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝
|
||||
- 詞庫資料:[小麥注音 McBopomofo](https://github.com/openvanilla/McBopomofo)(MIT License)
|
||||
- Windows 輸入法框架:[PIME](https://github.com/EasyIME/PIME)(LGPL 2.1,使用者自行安裝,本專案不包含其程式碼)
|
||||
- 英文字典:Webster's Second International Dictionary(公有領域,macOS 的 `/usr/share/dict/words`)
|
||||
- AI 選字語言模型:[CKIP GPT2 Tiny Chinese](https://github.com/ckiplab/ckip-transformers)(中研院詞庫小組,GPL-3.0,全文在 `licenses/GPL-3.0.txt`,Mac 版一併放在 app 的 Resources)
|
||||
- AI 選字語言模型:[CKIP GPT2 Tiny Chinese](https://github.com/ckiplab/ckip-transformers)(中研院詞庫小組,GPL-3.0,全文在 `licenses/GPL-3.0.txt`,並一起打包進三個平台的安裝檔)
|
||||
- 數值運算:[NumPy](https://numpy.org)(BSD 3-Clause,Windows 版附上、Linux 版安裝時下載)
|
||||
|
||||
完整的第三方授權聲明(含小麥注音 MIT 授權全文)見 [THIRD_PARTY_LICENSES.txt](THIRD_PARTY_LICENSES.txt),Mac 版放在 `AutoZhuyin.app/Contents/Resources/`,Windows 版放在 zip 根目錄和安裝後的 `autozhuyin` 資料夾。
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
/* Names shown in System Settings and the input menu. */
|
||||
CFBundleName = "自動注音";
|
||||
CFBundleDisplayName = "自動注音";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin" = "自動注音";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin.Auto" = "自動注音(中英自動)";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin.English" = "自動注音(英文)";
|
||||
+63
-2
@@ -15,9 +15,70 @@
|
||||
<key>CFBundlePackageType</key>
|
||||
<string>APPL</string>
|
||||
<key>CFBundleShortVersionString</key>
|
||||
<string>0.11.0</string>
|
||||
<string>0.13.2</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>19</string>
|
||||
<string>23</string>
|
||||
<key>ComponentInputModeDict</key>
|
||||
<dict>
|
||||
<key>tsInputModeListKey</key>
|
||||
<dict>
|
||||
<key>com.autozhuyin.inputmethod.AutoZhuyin.Auto</key>
|
||||
<dict>
|
||||
<key>TISInputSourceID</key>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin.Auto</string>
|
||||
<key>TISIntendedLanguage</key>
|
||||
<string>zh-Hant</string>
|
||||
<key>tsInputModeAlternateMenuIconFileKey</key>
|
||||
<string>icon.tiff</string>
|
||||
<key>tsInputModeCharacterRepertoireKey</key>
|
||||
<array>
|
||||
<string>Hant</string>
|
||||
</array>
|
||||
<key>tsInputModeDefaultStateKey</key>
|
||||
<true/>
|
||||
<key>tsInputModeIsVisibleKey</key>
|
||||
<true/>
|
||||
<key>tsInputModeMenuIconFileKey</key>
|
||||
<string>icon.tiff</string>
|
||||
<key>tsInputModePaletteIconFileKey</key>
|
||||
<string>icon.tiff</string>
|
||||
<key>tsInputModePrimaryInScriptKey</key>
|
||||
<true/>
|
||||
<key>tsInputModeScriptKey</key>
|
||||
<string>smTradChinese</string>
|
||||
</dict>
|
||||
<key>com.autozhuyin.inputmethod.AutoZhuyin.English</key>
|
||||
<dict>
|
||||
<key>TISInputSourceID</key>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin.English</string>
|
||||
<key>TISIntendedLanguage</key>
|
||||
<string>zh-Hant</string>
|
||||
<key>tsInputModeAlternateMenuIconFileKey</key>
|
||||
<string>icon-en.tiff</string>
|
||||
<key>tsInputModeCharacterRepertoireKey</key>
|
||||
<array>
|
||||
<string>Hant</string>
|
||||
</array>
|
||||
<key>tsInputModeDefaultStateKey</key>
|
||||
<true/>
|
||||
<key>tsInputModeIsVisibleKey</key>
|
||||
<true/>
|
||||
<key>tsInputModeMenuIconFileKey</key>
|
||||
<string>icon-en.tiff</string>
|
||||
<key>tsInputModePaletteIconFileKey</key>
|
||||
<string>icon-en.tiff</string>
|
||||
<key>tsInputModePrimaryInScriptKey</key>
|
||||
<false/>
|
||||
<key>tsInputModeScriptKey</key>
|
||||
<string>smTradChinese</string>
|
||||
</dict>
|
||||
</dict>
|
||||
<key>tsVisibleInputModeOrderedArrayKey</key>
|
||||
<array>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin.Auto</string>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin.English</string>
|
||||
</array>
|
||||
</dict>
|
||||
<key>InputMethodConnectionName</key>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
||||
<key>InputMethodServerControllerClass</key>
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
/* Names shown in System Settings and the input menu. */
|
||||
CFBundleName = "AutoZhuyin";
|
||||
CFBundleDisplayName = "AutoZhuyin";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin" = "AutoZhuyin";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin.Auto" = "AutoZhuyin (Auto)";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin.English" = "AutoZhuyin (English)";
|
||||
@@ -0,0 +1,6 @@
|
||||
/* Names shown in System Settings and the input menu. */
|
||||
CFBundleName = "自動注音";
|
||||
CFBundleDisplayName = "自動注音";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin" = "自動注音";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin.Auto" = "自動注音(中英自動)";
|
||||
"com.autozhuyin.inputmethod.AutoZhuyin.English" = "自動注音(英文)";
|
||||
@@ -1,3 +1,4 @@
|
||||
import Carbon
|
||||
import Cocoa
|
||||
import InputMethodKit
|
||||
import AutoZhuyinCore
|
||||
@@ -74,8 +75,11 @@ final class InputController: IMKInputController {
|
||||
otherModifiers: !flags.intersection([.command, .control, .option, .function]).isEmpty,
|
||||
time: event.timestamp)
|
||||
if tapped {
|
||||
let idle = engine.keys.isEmpty
|
||||
_ = engine.handle(.shiftTap)
|
||||
updateComposition(client)
|
||||
// English mode on/off: switch input mode so the menu-bar icon shows it.
|
||||
if idle { InputModes.select(english: engine.isEnglishMode) }
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -152,6 +156,18 @@ final class InputController: IMKInputController {
|
||||
}
|
||||
}
|
||||
|
||||
/// The input mode changed (menu bar, keyboard shortcut, or our own Shift tap).
|
||||
override func setValue(_ value: Any!, forTag tag: Int, client sender: Any!) {
|
||||
if tag == kTextServiceInputModePropertyTag, let id = value as? String {
|
||||
let english = InputModes.isEnglish(id)
|
||||
if english != engine.isEnglishMode {
|
||||
commitComposition(sender)
|
||||
engine.setEnglishMode(english)
|
||||
}
|
||||
}
|
||||
super.setValue(value, forTag: tag, client: sender)
|
||||
}
|
||||
|
||||
override func commitComposition(_ sender: Any!) {
|
||||
guard let client = sender as? IMKTextInput else { return }
|
||||
commit(engine.commitAll(), to: client)
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import Carbon
|
||||
import Foundation
|
||||
|
||||
/// The two input modes declared in Info.plist (ComponentInputModeDict). Each has its own
|
||||
/// menu-bar icon, so switching modes is how the icon shows "中英自動" vs "英文".
|
||||
enum InputModes {
|
||||
static let auto = "com.autozhuyin.inputmethod.AutoZhuyin.Auto"
|
||||
static let english = "com.autozhuyin.inputmethod.AutoZhuyin.English"
|
||||
|
||||
static func isEnglish(_ modeID: String) -> Bool { modeID == english }
|
||||
|
||||
private static func source(_ id: String) -> TISInputSource? {
|
||||
let filter = [kTISPropertyInputSourceID as String: id] as CFDictionary
|
||||
guard let list = TISCreateInputSourceList(filter, true)?.takeRetainedValue() as? [TISInputSource] else { return nil }
|
||||
return list.first
|
||||
}
|
||||
|
||||
private static func isEnabled(_ src: TISInputSource) -> Bool {
|
||||
guard let p = TISGetInputSourceProperty(src, kTISPropertyInputSourceIsEnabled) else { return false }
|
||||
return CFBooleanGetValue(Unmanaged<CFBoolean>.fromOpaque(p).takeUnretainedValue())
|
||||
}
|
||||
|
||||
/// Makes the given mode current. Returns false on failure.
|
||||
/// Only enables the mode if it is not enabled yet: TISEnableInputSource opens System
|
||||
/// Settings → Keyboard, even for a source that is already enabled.
|
||||
@discardableResult
|
||||
static func select(english: Bool) -> Bool {
|
||||
guard let src = source(english ? InputModes.english : InputModes.auto) else { return false }
|
||||
if !isEnabled(src) { TISEnableInputSource(src) }
|
||||
return TISSelectInputSource(src) == noErr
|
||||
}
|
||||
|
||||
/// Called by `--register`: enable both modes (if not yet) so they are in the input menu.
|
||||
static func enableAll() {
|
||||
for id in [auto, english] {
|
||||
if let src = source(id), !isEnabled(src) { TISEnableInputSource(src) }
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import InputMethodKit
|
||||
// in System Settings without logging out (used by the installer's postinstall script).
|
||||
if CommandLine.arguments.contains("--register") {
|
||||
let status = TISRegisterInputSource(Bundle.main.bundleURL as CFURL)
|
||||
if status == noErr { InputModes.enableAll() } // "中英自動" and "英文" (menu-bar icons)
|
||||
print(status == noErr ? "Registered \(Bundle.main.bundlePath)" : "TISRegisterInputSource failed: \(status)")
|
||||
exit(status == noErr ? 0 : 1)
|
||||
}
|
||||
|
||||
@@ -417,6 +417,16 @@ public final class CompositionEngine {
|
||||
}
|
||||
}
|
||||
|
||||
/// Turns English mode on or off directly (e.g. the input mode was switched in the menu
|
||||
/// bar). Commit any composition first.
|
||||
public func setEnglishMode(_ on: Bool) {
|
||||
guard on != isEnglishMode else { return }
|
||||
reset()
|
||||
isEnglishMode = on
|
||||
previousMode = on ? .literal : nil
|
||||
previousCommitTime = now()
|
||||
}
|
||||
|
||||
/// Forces the whole buffer into the other mode (a correction that is learned).
|
||||
private func flip() {
|
||||
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
|
||||
|
||||
@@ -50,8 +50,8 @@ The Windows version runs as a module inside PIME, which users install
|
||||
separately. No PIME code is included in this software.
|
||||
|
||||
|
||||
4. CKIP GPT2 Tiny Chinese — AI 選字語言模型 (language model, macOS)
|
||||
-------------------------------------------------------------------
|
||||
4. CKIP GPT2 Tiny Chinese — AI 選字語言模型 (language model)
|
||||
-----------------------------------------------------------
|
||||
https://huggingface.co/ckiplab/gpt2-tiny-chinese
|
||||
https://github.com/ckiplab/ckip-transformers
|
||||
Author: Mu Yang at CKIP, Academia Sinica (中央研究院 詞庫小組)
|
||||
@@ -62,3 +62,12 @@ lm-vocab.txt is its vocabulary (from bert-base-chinese). They are used unmodifie
|
||||
otherwise, to rerank conversion candidates ("AI 選字"). The source code of this input
|
||||
method, including the conversion script and the code that runs the model, is
|
||||
available from the project repository.
|
||||
|
||||
|
||||
5. NumPy — 數值運算 (Windows / Linux, for AI 選字)
|
||||
------------------------------------------------
|
||||
https://numpy.org — BSD 3-Clause License (and the licenses of the libraries it bundles,
|
||||
such as OpenBLAS). The Windows package includes unmodified NumPy wheels for 32-bit
|
||||
Python 3.8 and 3.15 under autozhuyin/vendor/; the Linux installer downloads the
|
||||
matching wheel from PyPI when the system has no NumPy. Each copy keeps its license
|
||||
files in its numpy-*.dist-info folder.
|
||||
|
||||
@@ -133,6 +133,16 @@ final class EngineTests: XCTestCase {
|
||||
return out
|
||||
}
|
||||
|
||||
/// The menu-bar input mode can switch English mode directly.
|
||||
func testSetEnglishMode() {
|
||||
let e = makeEngine()
|
||||
e.setEnglishMode(true)
|
||||
XCTAssertTrue(e.isEnglishMode)
|
||||
XCTAssertEqual(e.handle(.character("a")).handled, false) // goes to the app
|
||||
e.setEnglishMode(false)
|
||||
XCTAssertEqual(type("su3cl3 ", into: e), "你好")
|
||||
}
|
||||
|
||||
func testMixedSentence() {
|
||||
let e = makeEngine()
|
||||
// 我 + "hello world " + 你好
|
||||
|
||||
@@ -17,6 +17,10 @@ import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
# numpy for "AI 選字", downloaded by install.sh when the system has none.
|
||||
_VENDOR = os.path.join(HERE, "vendor", "cp%d%d" % sys.version_info[:2])
|
||||
if os.path.isdir(_VENDOR):
|
||||
sys.path.insert(1, _VENDOR)
|
||||
|
||||
import gi # noqa: E402
|
||||
|
||||
@@ -24,7 +28,9 @@ gi.require_version("IBus", "1.0")
|
||||
from gi.repository import GLib, GObject, IBus # noqa: E402
|
||||
|
||||
import autozhuyin # noqa: E402
|
||||
from autozhuyin.core.ai import AI, MODE_NAMES # noqa: E402
|
||||
from autozhuyin.core.history import UserHistory # noqa: E402
|
||||
from autozhuyin.core.reranker import MODES # noqa: E402
|
||||
from autozhuyin.core.classifier import LITERAL, Classifier, LearnedPreferences # noqa: E402
|
||||
from autozhuyin.core.engine import ( # noqa: E402
|
||||
BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, SHIFT_TAP, TAB, UP, CompositionEngine,
|
||||
@@ -50,6 +56,15 @@ except OSError:
|
||||
pass
|
||||
|
||||
_classifier = None
|
||||
_ai = None
|
||||
|
||||
|
||||
def shared_ai():
|
||||
""""AI 選字": settings and the language model, loaded in the background once."""
|
||||
global _ai
|
||||
if _ai is None:
|
||||
_ai = AI(_DATA, _USER_DIR)
|
||||
return _ai
|
||||
|
||||
|
||||
def _save_json(path, data):
|
||||
@@ -118,6 +133,7 @@ class AutoZhuyinEngine(IBus.Engine):
|
||||
if getattr(self, "engine", None) is not None:
|
||||
return
|
||||
self.engine = CompositionEngine(shared_classifier())
|
||||
self.engine.reranker = shared_ai().reranker
|
||||
self.shift_tap = ShiftTapDetector()
|
||||
self.table = IBus.LookupTable.new(self.engine.page_size, 0, True, False)
|
||||
self.table.set_orientation(IBus.Orientation.VERTICAL)
|
||||
@@ -127,8 +143,43 @@ class AutoZhuyinEngine(IBus.Engine):
|
||||
symbol=IBus.Text.new_from_string("自"),
|
||||
tooltip=IBus.Text.new_from_string("單按 Shift 切換英文模式"),
|
||||
sensitive=True, visible=True, state=IBus.PropState.UNCHECKED)
|
||||
# "AI 選字" menu: modes (radio), auto-pause (toggle), status, clear learned data.
|
||||
ai = shared_ai()
|
||||
self.ai_props = {}
|
||||
sub = IBus.PropList()
|
||||
for mode in MODES:
|
||||
prop = IBus.Property(
|
||||
key="ai." + mode, prop_type=IBus.PropType.RADIO,
|
||||
label=IBus.Text.new_from_string("AI 選字:" + MODE_NAMES[mode]),
|
||||
tooltip=IBus.Text.new_from_string(""), sensitive=True, visible=True,
|
||||
state=IBus.PropState.CHECKED if ai.reranker.mode == mode else IBus.PropState.UNCHECKED)
|
||||
self.ai_props[mode] = prop
|
||||
sub.append(prop)
|
||||
self.pause_prop = IBus.Property(
|
||||
key="ai.autopause", prop_type=IBus.PropType.TOGGLE,
|
||||
label=IBus.Text.new_from_string("系統忙碌時自動暫停 AI"),
|
||||
tooltip=IBus.Text.new_from_string(""), sensitive=True, visible=True,
|
||||
state=IBus.PropState.CHECKED if ai.auto_pause else IBus.PropState.UNCHECKED)
|
||||
self.status_prop = IBus.Property(
|
||||
key="ai.status", prop_type=IBus.PropType.NORMAL,
|
||||
label=IBus.Text.new_from_string(ai.status_text()),
|
||||
tooltip=IBus.Text.new_from_string(""), sensitive=False, visible=True,
|
||||
state=IBus.PropState.UNCHECKED)
|
||||
self.clear_prop = IBus.Property(
|
||||
key="clear", prop_type=IBus.PropType.NORMAL,
|
||||
label=IBus.Text.new_from_string("清除學習紀錄"),
|
||||
tooltip=IBus.Text.new_from_string(""), sensitive=True, visible=True,
|
||||
state=IBus.PropState.UNCHECKED)
|
||||
for prop in (self.pause_prop, self.status_prop, self.clear_prop):
|
||||
sub.append(prop)
|
||||
self.ai_menu = IBus.Property(
|
||||
key="ai", prop_type=IBus.PropType.MENU,
|
||||
label=IBus.Text.new_from_string("設定"),
|
||||
tooltip=IBus.Text.new_from_string("AI 選字與學習紀錄"),
|
||||
sensitive=True, visible=True, state=IBus.PropState.UNCHECKED, sub_props=sub)
|
||||
self.props = IBus.PropList()
|
||||
self.props.append(self.mode_prop)
|
||||
self.props.append(self.ai_menu)
|
||||
|
||||
# --- keys ---
|
||||
|
||||
@@ -211,12 +262,38 @@ class AutoZhuyinEngine(IBus.Engine):
|
||||
self._update(None)
|
||||
return True
|
||||
|
||||
# --- settings menu ---
|
||||
|
||||
def do_property_activate(self, prop_name, state):
|
||||
self._setup()
|
||||
ai = shared_ai()
|
||||
if prop_name.startswith("ai.") and prop_name[3:] in MODES:
|
||||
if state == IBus.PropState.CHECKED:
|
||||
ai.set_mode(prop_name[3:])
|
||||
elif prop_name == "ai.autopause":
|
||||
ai.set_auto_pause(state == IBus.PropState.CHECKED)
|
||||
elif prop_name == "clear":
|
||||
shared_classifier().learned.clear()
|
||||
shared_classifier().history.clear()
|
||||
self._refresh_ai_props()
|
||||
|
||||
def _refresh_ai_props(self):
|
||||
ai = shared_ai()
|
||||
for mode, prop in self.ai_props.items():
|
||||
prop.set_state(IBus.PropState.CHECKED if ai.reranker.mode == mode else IBus.PropState.UNCHECKED)
|
||||
self.update_property(prop)
|
||||
self.pause_prop.set_state(IBus.PropState.CHECKED if ai.auto_pause else IBus.PropState.UNCHECKED)
|
||||
self.update_property(self.pause_prop)
|
||||
self.status_prop.set_label(IBus.Text.new_from_string(ai.status_text()))
|
||||
self.update_property(self.status_prop)
|
||||
|
||||
# --- lifecycle ---
|
||||
|
||||
def do_focus_in(self):
|
||||
self._setup()
|
||||
self.register_properties(self.props)
|
||||
self._update_mode()
|
||||
self._refresh_ai_props()
|
||||
|
||||
def do_focus_out(self):
|
||||
self._setup()
|
||||
@@ -268,6 +345,9 @@ class AutoZhuyinEngine(IBus.Engine):
|
||||
english = self.engine.english_mode
|
||||
self.mode_prop.set_label(IBus.Text.new_from_string("英文" if english else "中英自動"))
|
||||
self.mode_prop.set_symbol(IBus.Text.new_from_string("英" if english else "自"))
|
||||
icon = os.path.join(HERE, "icon-en.png" if english else "icon.png")
|
||||
if os.path.exists(icon):
|
||||
self.mode_prop.set_icon(icon) # panels that show property icons
|
||||
self.update_property(self.mode_prop)
|
||||
|
||||
|
||||
|
||||
@@ -33,13 +33,23 @@
|
||||
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字;
|
||||
選過的字會被記住,下次優先出現
|
||||
Esc 取消組字
|
||||
單按 Shift 切換英文模式(按鍵直接送出),再單按一次回到中英自動判斷
|
||||
單按 Shift 切換英文模式(按鍵直接送出,狀態圖示變成「英」),再單按一次回到中英自動判斷
|
||||
Shift+字母 打大寫字母(也會強制英文)
|
||||
` + 標點 強制全形標點,例如 `< 或 `, = , `. = 。 `/ = ? `; = ;
|
||||
|
||||
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
|
||||
|
||||
|
||||
AI 選字
|
||||
-------
|
||||
用中研院 CKIP 的繁體中文小型語言模型,看前後文挑同音字(他跑「得」很快、請你「再」說一次),
|
||||
完全在本機運算。需要 numpy:系統沒有的話,install.sh 會從 PyPI 下載符合的版本(約 15 MB)。
|
||||
在 IBus/系統匣的輸入法選單「設定」裡可以選:
|
||||
AI 選字:關閉/省電/標準(預設)/高品質
|
||||
系統忙碌時自動暫停 AI(預設開啟:CPU 很忙或過熱時暫停,省電設定或電量低時改用省電)
|
||||
設定存在 ~/.config/autozhuyin/settings.json。
|
||||
|
||||
|
||||
切換與移除
|
||||
----------
|
||||
暫時改回一般鍵盤: ibus engine xkb:us::eng
|
||||
|
||||
@@ -23,13 +23,31 @@ fi
|
||||
|
||||
# 2. Copy the program.
|
||||
pkill -f "autozhuyin/autozhuyin_ibus.py" 2>/dev/null || true
|
||||
VENDOR_BACKUP=""
|
||||
if [ -d "$DEST/vendor" ]; then # keep numpy downloaded by an earlier install
|
||||
VENDOR_BACKUP="$(mktemp -d)"
|
||||
mv "$DEST/vendor" "$VENDOR_BACKUP/"
|
||||
fi
|
||||
rm -rf "$DEST"
|
||||
mkdir -p "$DEST"
|
||||
cp -R autozhuyin autozhuyin_ibus.py version.txt "$DEST/"
|
||||
[ -f icon.png ] && cp icon.png "$DEST/"
|
||||
[ -f THIRD_PARTY_LICENSES.txt ] && cp THIRD_PARTY_LICENSES.txt "$DEST/"
|
||||
cp -R autozhuyin autozhuyin_ibus.py version.txt tools "$DEST/"
|
||||
for f in icon.png icon-en.png THIRD_PARTY_LICENSES.txt GPL-3.0.txt; do [ -f "$f" ] && cp "$f" "$DEST/"; done
|
||||
[ -n "$VENDOR_BACKUP" ] && mv "$VENDOR_BACKUP/vendor" "$DEST/" && rmdir "$VENDOR_BACKUP"
|
||||
chmod +x "$DEST/autozhuyin_ibus.py"
|
||||
|
||||
# 2b. numpy for "AI 選字": use the system's, or download the matching version from PyPI.
|
||||
TAG="$("$PY" -c 'import sys; print("cp%d%d" % sys.version_info[:2])')"
|
||||
if "$PY" -c 'import numpy' 2>/dev/null; then
|
||||
echo "AI 選字:使用系統的 numpy。"
|
||||
elif [ -d "$DEST/vendor/$TAG/numpy" ]; then
|
||||
echo "AI 選字:使用先前下載的 numpy。"
|
||||
else
|
||||
echo "AI 選字需要 numpy,正在從 PyPI 下載(約 15 MB)…"
|
||||
if ! "$PY" -I "$DEST/tools/fetch_numpy.py" "$DEST/vendor" auto auto; then
|
||||
echo "下載 numpy 失敗:輸入法照常可用,只是沒有 AI 選字。之後可以重新執行 install.sh。"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 3. Start with the desktop session: registers the engine with IBus and selects it.
|
||||
mkdir -p "$AUTOSTART"
|
||||
cat > "$AUTOSTART/autozhuyin.desktop" <<DESKTOP
|
||||
|
||||
@@ -54,9 +54,11 @@ class LookupTable:
|
||||
|
||||
|
||||
class Property:
|
||||
def __init__(self, **kw): self.label = kw["label"].text
|
||||
def __init__(self, **kw): self.key, self.label, self.state = kw["key"], kw["label"].text, kw.get("state", 0)
|
||||
def set_label(self, t): self.label = t.text
|
||||
def set_symbol(self, t): pass
|
||||
def set_state(self, s): self.state = s
|
||||
def set_icon(self, path): self.icon = path
|
||||
|
||||
|
||||
class Engine:
|
||||
@@ -71,7 +73,9 @@ class Engine:
|
||||
def update_lookup_table(self, table, visible): self.shown = list(table.items)
|
||||
def hide_lookup_table(self): self.shown = None
|
||||
def register_properties(self, p): pass
|
||||
def update_property(self, p): self.mode_label = p.label
|
||||
def update_property(self, p):
|
||||
if p.key == "mode":
|
||||
self.mode_label = p.label
|
||||
|
||||
|
||||
IBus.Text, IBus.AttrList, IBus.LookupTable, IBus.Property, IBus.Engine = Text, AttrList, LookupTable, Property, Engine
|
||||
@@ -80,8 +84,8 @@ IBus.attr_underline_new = lambda kind, a, b: ("underline", a, b)
|
||||
IBus.attr_foreground_new = lambda color, a, b: ("color", a, b)
|
||||
IBus.AttrUnderline = types.SimpleNamespace(SINGLE=1)
|
||||
IBus.Orientation = types.SimpleNamespace(VERTICAL=1)
|
||||
IBus.PropType = types.SimpleNamespace(NORMAL=0)
|
||||
IBus.PropState = types.SimpleNamespace(UNCHECKED=0)
|
||||
IBus.PropType = types.SimpleNamespace(NORMAL=0, TOGGLE=1, RADIO=2, MENU=3)
|
||||
IBus.PropState = types.SimpleNamespace(UNCHECKED=0, CHECKED=1)
|
||||
IBus.PreeditFocusMode = types.SimpleNamespace(COMMIT=1)
|
||||
|
||||
gi = types.ModuleType("gi")
|
||||
@@ -173,6 +177,19 @@ class IBusEngineTests(unittest.TestCase):
|
||||
press(self.e, "su3 \n")
|
||||
self.assertTrue(self.e.committed.endswith(second))
|
||||
|
||||
def test_ai_menu(self):
|
||||
e = self.e
|
||||
self.assertEqual(e.ai_props["standard"].state, 1)
|
||||
e.do_property_activate("ai.saver", 1)
|
||||
self.assertEqual(autozhuyin_ibus.shared_ai().reranker.mode, "saver")
|
||||
self.assertEqual(e.ai_props["saver"].state, 1)
|
||||
self.assertEqual(e.ai_props["standard"].state, 0)
|
||||
e.do_property_activate("ai.autopause", 0)
|
||||
self.assertFalse(autozhuyin_ibus.shared_ai().auto_pause)
|
||||
self.assertTrue(e.status_prop.label)
|
||||
e.do_property_activate("ai.standard", 1)
|
||||
e.do_property_activate("ai.autopause", 1)
|
||||
|
||||
def test_focus_out_resets(self):
|
||||
press(self.e, "su3")
|
||||
self.e.do_focus_out()
|
||||
|
||||
@@ -18,6 +18,8 @@ rm -rf "$APP"
|
||||
mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
|
||||
cp "$BIN" "$APP/Contents/MacOS/AutoZhuyin"
|
||||
cp Resources/Info.plist "$APP/Contents/Info.plist"
|
||||
# Localized names of the input method and its modes (System Settings, input menu).
|
||||
cp -R Resources/*.lproj "$APP/Contents/Resources/"
|
||||
cp Resources/lexicon.txt Resources/english-common.txt Resources/classifier-weights.json "$APP/Contents/Resources/"
|
||||
cp THIRD_PARTY_LICENSES.txt "$APP/Contents/Resources/"
|
||||
# "AI 選字" language model (CKIP gpt2-tiny-chinese, GPL-3.0), fetched and converted once.
|
||||
@@ -31,6 +33,9 @@ if [ ! -f build/icon.tiff ]; then
|
||||
xcrun swift scripts/make-icon.swift build/icon.tiff
|
||||
fi
|
||||
cp build/icon.tiff "$APP/Contents/Resources/icon.tiff"
|
||||
# English-mode icon ("A"), shown in the menu bar while English mode is on.
|
||||
[ -f build/icon-en.tiff ] || xcrun swift scripts/make-icon.swift build/icon-en.tiff A
|
||||
cp build/icon-en.tiff "$APP/Contents/Resources/icon-en.tiff"
|
||||
|
||||
codesign --force --sign - "$APP"
|
||||
echo "Built $APP"
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
// Renders the menu-bar icon ("注" with a small "A") to a TIFF.
|
||||
// Usage: swift scripts/make-icon.swift <output.tiff>
|
||||
// Renders a menu-bar icon (one character, "注" by default; "A" for English mode) to a TIFF.
|
||||
// Usage: swift scripts/make-icon.swift <output.tiff> [character]
|
||||
import AppKit
|
||||
|
||||
let out = CommandLine.arguments.count > 1 ? CommandLine.arguments[1] : "icon.tiff"
|
||||
let glyph = CommandLine.arguments.count > 2 ? CommandLine.arguments[2] : "注"
|
||||
let reps: [NSBitmapImageRep] = [16, 32].map { size in
|
||||
let rep = NSBitmapImageRep(bitmapDataPlanes: nil, pixelsWide: size, pixelsHigh: size,
|
||||
bitsPerSample: 8, samplesPerPixel: 4, hasAlpha: true, isPlanar: false,
|
||||
@@ -11,7 +12,7 @@ let reps: [NSBitmapImageRep] = [16, 32].map { size in
|
||||
NSGraphicsContext.saveGraphicsState()
|
||||
NSGraphicsContext.current = NSGraphicsContext(bitmapImageRep: rep)
|
||||
let font = NSFont.systemFont(ofSize: 13, weight: .semibold)
|
||||
let text = NSAttributedString(string: "注", attributes: [.font: font, .foregroundColor: NSColor.black])
|
||||
let text = NSAttributedString(string: glyph, attributes: [.font: font, .foregroundColor: NSColor.black])
|
||||
let s = text.size()
|
||||
text.draw(at: NSPoint(x: (16 - s.width) / 2, y: (16 - s.height) / 2))
|
||||
NSGraphicsContext.restoreGraphicsState()
|
||||
|
||||
@@ -8,6 +8,7 @@ VERSION="$(python3 -I -c 'import json,sys; print(json.load(open(sys.argv[1]))["v
|
||||
NAME="AutoZhuyin-Linux-$VERSION"
|
||||
OUT="build/$NAME"
|
||||
|
||||
[ -s build/lm/lm.bin ] || scripts/fetch-lm.sh >/dev/null # "AI 選字" model
|
||||
python3 -I windows/tools/build_data.py
|
||||
(cd windows && python3 -m unittest discover -q tests)
|
||||
python3 -m unittest discover -q -s linux/tests
|
||||
@@ -21,9 +22,13 @@ mkdir -p "$OUT/autozhuyin"
|
||||
rsync -a --exclude __pycache__ --exclude '*.pyc' \
|
||||
windows/autozhuyin/__init__.py windows/autozhuyin/core windows/autozhuyin/data "$OUT/autozhuyin/"
|
||||
cp linux/autozhuyin_ibus.py linux/installer/install.sh linux/installer/uninstall.sh linux/installer/README.txt \
|
||||
THIRD_PARTY_LICENSES.txt "$OUT/"
|
||||
THIRD_PARTY_LICENSES.txt licenses/GPL-3.0.txt "$OUT/"
|
||||
mkdir -p "$OUT/tools"
|
||||
cp windows/tools/fetch_numpy.py "$OUT/tools/" # install.sh downloads numpy if the system has none
|
||||
echo "$VERSION" > "$OUT/version.txt"
|
||||
sips -s format png -z 64 64 build/icon.tiff --out "$OUT/icon.png" >/dev/null
|
||||
[ -f build/icon-en.tiff ] || DEVELOPER_DIR="${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}" xcrun swift scripts/make-icon.swift build/icon-en.tiff A
|
||||
sips -s format png -z 64 64 build/icon-en.tiff --out "$OUT/icon-en.png" >/dev/null
|
||||
chmod +x "$OUT/install.sh" "$OUT/uninstall.sh" "$OUT/autozhuyin_ibus.py"
|
||||
|
||||
# Plain ustar without macOS extended attributes, so GNU tar doesn't warn.
|
||||
|
||||
@@ -8,6 +8,7 @@ VERSION="$(python3 -I -c 'import json,sys; print(json.load(open(sys.argv[1]))["v
|
||||
NAME="AutoZhuyin-Windows-$VERSION"
|
||||
OUT="build/$NAME"
|
||||
|
||||
[ -s build/lm/lm.bin ] || scripts/fetch-lm.sh >/dev/null # "AI 選字" model
|
||||
python3 -I windows/tools/build_data.py
|
||||
(cd windows && python3 -m unittest discover -q tests)
|
||||
|
||||
@@ -17,6 +18,10 @@ if [ ! -f build/icon.tiff ]; then
|
||||
fi
|
||||
sips -s format png -z 32 32 build/icon.tiff --out build/icon-32.png >/dev/null
|
||||
python3 -I windows/tools/png_to_ico.py build/icon-32.png windows/autozhuyin/icon.ico
|
||||
# English-mode icon ("A") for the taskbar while English mode is on.
|
||||
[ -f build/icon-en.tiff ] || DEVELOPER_DIR="${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}" xcrun swift scripts/make-icon.swift build/icon-en.tiff A
|
||||
sips -s format png -z 32 32 build/icon-en.tiff --out build/icon-en-32.png >/dev/null
|
||||
python3 -I windows/tools/png_to_ico.py build/icon-en-32.png windows/autozhuyin/icon-en.ico
|
||||
|
||||
rm -rf "$OUT" "$OUT.zip"
|
||||
mkdir -p "$OUT"
|
||||
@@ -29,6 +34,14 @@ done
|
||||
# Third-party notices: next to the installer and inside the installed folder.
|
||||
sed 's/$/\r/' THIRD_PARTY_LICENSES.txt > "$OUT/THIRD_PARTY_LICENSES.txt"
|
||||
cp "$OUT/THIRD_PARTY_LICENSES.txt" "$OUT/autozhuyin/"
|
||||
sed 's/$/\r/' licenses/GPL-3.0.txt > "$OUT/autozhuyin/GPL-3.0.txt"
|
||||
|
||||
# numpy for "AI 選字", for both Pythons PIME ships (1.3: 3.8, 2.x: 3.15; both 32-bit).
|
||||
for tag in cp38 cp315; do
|
||||
[ -d "build/numpy-vendor/$tag" ] || python3 -I windows/tools/fetch_numpy.py build/numpy-vendor "$tag" win32
|
||||
done
|
||||
mkdir -p "$OUT/autozhuyin/vendor"
|
||||
cp -R build/numpy-vendor/cp38 build/numpy-vendor/cp315 "$OUT/autozhuyin/vendor/"
|
||||
|
||||
(cd build && zip -qr -X "$NAME.zip" "$NAME")
|
||||
echo "Built build/$NAME.zip"
|
||||
|
||||
@@ -6,12 +6,21 @@ tests); this file only translates PIME key events and replies.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
from keycodes import * # noqa: F401,F403 (VK_* constants, provided by PIME)
|
||||
from textService import TextService
|
||||
# numpy for "AI 選字" is bundled per Python version (PIME 1.3 has 3.8, PIME 2 has 3.15).
|
||||
_VENDOR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "vendor",
|
||||
"cp%d%d" % sys.version_info[:2])
|
||||
if os.path.isdir(_VENDOR) and _VENDOR not in sys.path:
|
||||
sys.path.insert(0, _VENDOR)
|
||||
|
||||
from keycodes import * # noqa: F401,F403 (VK_* constants, provided by PIME)
|
||||
from textService import COMMAND_LEFT_CLICK, COMMAND_MENU, TextService # noqa: F401
|
||||
|
||||
from .core.ai import AI, MODE_NAMES
|
||||
from .core.history import UserHistory
|
||||
from .core.reranker import MODES
|
||||
from .core.classifier import Classifier, LearnedPreferences
|
||||
from .core.engine import BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, SHIFT_TAP, TAB, UP, CompositionEngine
|
||||
from .core.english import EnglishModel
|
||||
@@ -26,6 +35,12 @@ _USER_PHRASES = os.path.join(_USER_DIR, "user-phrases.txt")
|
||||
_HISTORY_FILE = os.path.join(_USER_DIR, "user-history.json")
|
||||
|
||||
_classifier = None
|
||||
_ai = None
|
||||
|
||||
ID_MODE_ICON, ID_AUTO_PAUSE, ID_STATUS, ID_CLEAR = 1, 2, 3, 4
|
||||
ID_AI_MODE = {mode: 10 + i for i, mode in enumerate(MODES)} # 10..13
|
||||
_ICON = os.path.join(_HERE, "icon.ico")
|
||||
_ICON_EN = os.path.join(_HERE, "icon-en.ico") # shown while English mode is on
|
||||
|
||||
|
||||
def _save_json(path, data):
|
||||
@@ -72,6 +87,14 @@ def _shared_classifier():
|
||||
return _classifier
|
||||
|
||||
|
||||
def _shared_ai():
|
||||
""""AI 選字": settings and the language model, loaded in the background once."""
|
||||
global _ai
|
||||
if _ai is None:
|
||||
_ai = AI(_DATA, _USER_DIR)
|
||||
return _ai
|
||||
|
||||
|
||||
_SPECIAL = {
|
||||
VK_BACK: BACKSPACE, VK_RETURN: ENTER, VK_ESCAPE: ESCAPE, VK_TAB: TAB, # noqa: F405
|
||||
VK_UP: UP, VK_DOWN: DOWN, VK_LEFT: LEFT, VK_RIGHT: RIGHT, # noqa: F405
|
||||
@@ -83,6 +106,7 @@ class AutoZhuyinTextService(TextService):
|
||||
TextService.__init__(self, client)
|
||||
self.engine = CompositionEngine(_shared_classifier())
|
||||
self.shift_tap = ShiftTapDetector()
|
||||
self.engine.reranker = _shared_ai().reranker
|
||||
|
||||
# --- lifecycle ---
|
||||
|
||||
@@ -90,11 +114,68 @@ class AutoZhuyinTextService(TextService):
|
||||
TextService.onActivate(self)
|
||||
self.setSelKeys("123456789")
|
||||
self.customizeUI(candFontSize=16, candPerRow=1, candUseCursor=True)
|
||||
# Taskbar IME icon (Windows 8+) and language bar button; both open the settings menu.
|
||||
self._icon_english = self.engine.english_mode
|
||||
self.addButton("windows-mode-icon", icon=self._mode_icon(), tooltip=self._mode_tooltip(),
|
||||
commandId=ID_MODE_ICON)
|
||||
self.addButton("autozhuyin-settings", icon=_ICON, tooltip="自動注音設定", type="menu")
|
||||
|
||||
def onDeactivate(self):
|
||||
self.engine.reset()
|
||||
self.removeButton("windows-mode-icon")
|
||||
self.removeButton("autozhuyin-settings")
|
||||
TextService.onDeactivate(self)
|
||||
|
||||
# --- taskbar icon: "注" for 中英自動, "A" for English mode ---
|
||||
|
||||
def _mode_icon(self):
|
||||
return _ICON_EN if self.engine.english_mode and os.path.exists(_ICON_EN) else _ICON
|
||||
|
||||
def _mode_tooltip(self):
|
||||
return ("英文模式" if self.engine.english_mode else "中英自動") + "(左鍵切換,右鍵設定)"
|
||||
|
||||
def _update_mode_icon(self):
|
||||
if getattr(self, "_icon_english", None) == self.engine.english_mode:
|
||||
return
|
||||
self._icon_english = self.engine.english_mode
|
||||
self.changeButton("windows-mode-icon", icon=self._mode_icon(), tooltip=self._mode_tooltip())
|
||||
|
||||
# --- menu (AI 選字 settings) ---
|
||||
|
||||
def onMenu(self, buttonId):
|
||||
if buttonId not in ("windows-mode-icon", "autozhuyin-settings"):
|
||||
return None
|
||||
ai = _shared_ai()
|
||||
items = [{"text": "AI 選字:" + MODE_NAMES[m], "id": ID_AI_MODE[m], "checked": ai.reranker.mode == m}
|
||||
for m in MODES]
|
||||
items += [
|
||||
{},
|
||||
{"text": "系統忙碌時自動暫停 AI", "id": ID_AUTO_PAUSE, "checked": ai.auto_pause},
|
||||
{"text": ai.status_text(), "id": ID_STATUS, "enabled": False},
|
||||
{},
|
||||
{"text": "清除學習紀錄", "id": ID_CLEAR},
|
||||
]
|
||||
return items
|
||||
|
||||
def onCommand(self, commandId, commandType):
|
||||
ai = _shared_ai()
|
||||
if commandId == ID_MODE_ICON and commandType == COMMAND_LEFT_CLICK and not self.engine.keys:
|
||||
self.engine.handle(SHIFT_TAP) # left click: English mode on / off
|
||||
self._update_mode_icon()
|
||||
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
|
||||
return
|
||||
for mode, cid in ID_AI_MODE.items():
|
||||
if commandId == cid:
|
||||
ai.set_mode(mode)
|
||||
self.showMessage("AI 選字:" + MODE_NAMES[mode], 1)
|
||||
return
|
||||
if commandId == ID_AUTO_PAUSE:
|
||||
ai.set_auto_pause(not ai.auto_pause)
|
||||
elif commandId == ID_CLEAR:
|
||||
_shared_classifier().learned.clear()
|
||||
_shared_classifier().history.clear()
|
||||
self.showMessage("已清除學習紀錄", 1)
|
||||
|
||||
def onCompositionTerminated(self, forced):
|
||||
# The app ended the composition (focus change, mouse click): the shown text stays.
|
||||
self.engine.reset()
|
||||
@@ -155,6 +236,7 @@ class AutoZhuyinTextService(TextService):
|
||||
return False
|
||||
self.engine.handle(SHIFT_TAP)
|
||||
self._update(None)
|
||||
self._update_mode_icon()
|
||||
if not self.engine.keys:
|
||||
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
|
||||
return True
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
""""AI 選字" for the Windows and Linux shells: settings, background model loading,
|
||||
system-load detection and the status line shown in the menu."""
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
|
||||
from . import lm
|
||||
from .reranker import (MODES, NORMAL, OFF, ST_ACTIVE, ST_BUSY, ST_LOADING, ST_OFF, ST_SLOW,
|
||||
STANDARD, Reranker)
|
||||
from .system_monitor import REDUCED, SystemMonitor
|
||||
|
||||
MODE_NAMES = {OFF: "關閉", "saver": "省電", STANDARD: "標準", "quality": "高品質"}
|
||||
|
||||
|
||||
class AI:
|
||||
def __init__(self, data_dir, user_dir, monitor=None, background=True):
|
||||
self.model_path = os.path.join(data_dir, "lm.bin")
|
||||
self.vocab_path = os.path.join(data_dir, "lm-vocab.txt")
|
||||
self.settings_path = os.path.join(user_dir, "settings.json")
|
||||
self.monitor = monitor or SystemMonitor()
|
||||
self.background = background
|
||||
settings = self._load_settings()
|
||||
mode = settings.get("ai_mode", STANDARD)
|
||||
self.auto_pause = bool(settings.get("ai_auto_pause", True))
|
||||
self.reranker = Reranker(mode if mode in MODES else STANDARD)
|
||||
self.reranker.system_state = lambda: self.monitor.state() if self.auto_pause else NORMAL
|
||||
self.error = None # why the model cannot be used, if it cannot
|
||||
self._loading = False
|
||||
self.load_model()
|
||||
|
||||
# --- settings ---
|
||||
|
||||
def _load_settings(self):
|
||||
try:
|
||||
with open(self.settings_path, encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
return data if isinstance(data, dict) else {}
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
|
||||
def _save_settings(self):
|
||||
data = self._load_settings()
|
||||
data.update(ai_mode=self.reranker.mode, ai_auto_pause=self.auto_pause)
|
||||
try:
|
||||
os.makedirs(os.path.dirname(self.settings_path), exist_ok=True)
|
||||
with open(self.settings_path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=1)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def set_mode(self, mode):
|
||||
if mode not in MODES:
|
||||
return
|
||||
self.reranker.mode = mode
|
||||
if mode == OFF:
|
||||
self.reranker.model = None # free the memory
|
||||
else:
|
||||
self.load_model()
|
||||
self._save_settings()
|
||||
|
||||
def set_auto_pause(self, on):
|
||||
self.auto_pause = bool(on)
|
||||
self._save_settings()
|
||||
|
||||
# --- model ---
|
||||
|
||||
def load_model(self):
|
||||
if self.reranker.mode == OFF or self.reranker.model is not None or self._loading or self.error:
|
||||
return
|
||||
if not lm.available():
|
||||
self.error = "需要 numpy"
|
||||
return
|
||||
if not (os.path.exists(self.model_path) and os.path.exists(self.vocab_path)):
|
||||
self.error = "AI 模型未安裝"
|
||||
return
|
||||
self._loading = True
|
||||
|
||||
def work():
|
||||
try:
|
||||
model = lm.GPT2LanguageModel(self.model_path, self.vocab_path)
|
||||
except Exception as e: # noqa: BLE001 — any failure just disables AI
|
||||
self.error = f"AI 模型載入失敗:{e}"
|
||||
model = None
|
||||
self._loading = False
|
||||
if self.reranker.mode != OFF:
|
||||
self.reranker.model = model
|
||||
|
||||
if self.background:
|
||||
threading.Thread(target=work, daemon=True).start()
|
||||
else:
|
||||
work()
|
||||
|
||||
def status_text(self):
|
||||
if self.error:
|
||||
return self.error
|
||||
self.reranker.effective_mode() # refresh the status
|
||||
st = self.reranker.status
|
||||
if st == ST_OFF:
|
||||
return "AI 選字已關閉"
|
||||
if st == ST_LOADING:
|
||||
return "AI 模型載入中…"
|
||||
if st == ST_BUSY:
|
||||
return "AI 已暫停:系統忙碌或過熱"
|
||||
if st == ST_SLOW:
|
||||
return "AI 已暫停:運算太慢,一分鐘後再試"
|
||||
if st == ST_ACTIVE and self.auto_pause and self.monitor.state() == REDUCED:
|
||||
return "AI 使用中(省電模式)"
|
||||
return "AI 使用中"
|
||||
@@ -65,6 +65,8 @@ class CompositionEngine:
|
||||
self.inserting = []
|
||||
# Index of a held space (see HOLD_MIN_SCORE), or None.
|
||||
self.held = None
|
||||
self.reranker = None # "AI 選字" (reranker.Reranker); None = lexicon only
|
||||
self._recent_text = "" # last characters committed: context for the reranker
|
||||
self._previous_mode = None
|
||||
self._previous_time = float("-inf")
|
||||
self._cache_key = None
|
||||
@@ -159,8 +161,15 @@ class CompositionEngine:
|
||||
def _conversion(self, seg, final):
|
||||
syl, pending = self._convertible(seg, final)
|
||||
fx = {k: v for k, v in self.fixed.get(seg[0], {}).items() if k[1] <= len(syl)}
|
||||
if self.reranker is not None:
|
||||
chosen = self.reranker.choose(self.converter, syl, fx, self._rerank_context())
|
||||
if chosen is not None:
|
||||
return chosen, pending
|
||||
return self.converter.convert(syl, fx), pending
|
||||
|
||||
def _rerank_context(self):
|
||||
return self._recent_text if self.now() - self._previous_time < self.context_window else ""
|
||||
|
||||
def _render(self, seg, final):
|
||||
start, end, mode = seg
|
||||
if mode == LITERAL:
|
||||
@@ -556,6 +565,7 @@ class CompositionEngine:
|
||||
segs = segs if segs is not None else self.segments
|
||||
text = "".join(self._render(s, True) for s in segs)
|
||||
self._remember_picks(segs)
|
||||
self._recent_text = (self._rerank_context() + text)[-20:]
|
||||
if self.forced_mode:
|
||||
self.classifier.learned.record("".join(self.keys), self.forced_mode)
|
||||
self._previous_mode = segs[-1][2] if segs else None
|
||||
|
||||
@@ -130,6 +130,28 @@ class Converter:
|
||||
j = seg[0][0]
|
||||
return segs[::-1]
|
||||
|
||||
def kbest(self, syllables, k, fixed=None):
|
||||
"""The k best paths [(log10 score, [((start, end), text)])], best first."""
|
||||
fixed = fixed or {}
|
||||
n = len(syllables)
|
||||
if n == 0 or k <= 0:
|
||||
return []
|
||||
max_len = max(1, self.lexicon.max_phrase_length)
|
||||
beams = [[] for _ in range(n + 1)]
|
||||
beams[0] = [(0.0, [])]
|
||||
for j in range(1, n + 1):
|
||||
cand = []
|
||||
for length in range(1, min(max_len, j) + 1):
|
||||
i = j - length
|
||||
if not beams[i]:
|
||||
continue
|
||||
for text, score in self._options(syllables, i, j, fixed):
|
||||
for s, segs in beams[i]:
|
||||
cand.append((s + score, segs + [((i, j), text)]))
|
||||
cand.sort(key=lambda e: -e[0])
|
||||
beams[j] = cand[:k]
|
||||
return beams[n]
|
||||
|
||||
def candidates(self, syllables, end):
|
||||
"""[((start, end), text)] for phrases ending at syllable `end`, longest first."""
|
||||
if end <= 0 or end > len(syllables):
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
"""GPT-2 language model for "AI 選字" (numpy). Port of LanguageModel.swift.
|
||||
|
||||
Reads the "AZLM" file made by training/lm/export_lm.py (CKIP gpt2-tiny-chinese, GPL-3.0,
|
||||
float16 weights, expanded to float32 in memory). numpy is optional: when it cannot be
|
||||
imported, `available()` is False and the input method works without AI.
|
||||
"""
|
||||
import json
|
||||
import math
|
||||
import struct
|
||||
|
||||
try:
|
||||
import numpy as np
|
||||
except Exception: # ImportError, or a numpy build for another Python / architecture
|
||||
np = None
|
||||
|
||||
CLS, UNK = 101, 100
|
||||
|
||||
|
||||
def available():
|
||||
return np is not None
|
||||
|
||||
|
||||
class GPT2LanguageModel:
|
||||
max_tokens = 64
|
||||
|
||||
def __init__(self, model_path, vocab_path):
|
||||
if np is None:
|
||||
raise RuntimeError("numpy is not available")
|
||||
with open(model_path, "rb") as f:
|
||||
raw = f.read()
|
||||
if raw[:4] != b"AZLM":
|
||||
raise ValueError("not an AZLM file")
|
||||
n = struct.unpack("<I", raw[4:8])[0]
|
||||
header = json.loads(raw[8:8 + n])
|
||||
data = np.frombuffer(raw, dtype="<f2", offset=8 + n).astype(np.float32)
|
||||
self.n_layer, self.n_head = header["n_layer"], header["n_head"]
|
||||
self.d, self.eps = header["n_embd"], header["eps"]
|
||||
self.w = {}
|
||||
for name, shape, offset in header["tensors"]:
|
||||
size = int(np.prod(shape))
|
||||
self.w[name] = data[offset:offset + size].reshape(shape)
|
||||
self.vocab = {}
|
||||
with open(vocab_path, encoding="utf-8") as f:
|
||||
for i, line in enumerate(f):
|
||||
token = line.rstrip("\n")
|
||||
if len(token) == 1:
|
||||
self.vocab.setdefault(token, i)
|
||||
|
||||
def ids(self, text):
|
||||
return [self.vocab.get(c, UNK) for c in text]
|
||||
|
||||
def _ln(self, x, name):
|
||||
mu = x.mean(-1, keepdims=True)
|
||||
var = ((x - mu) ** 2).mean(-1, keepdims=True)
|
||||
return (x - mu) / np.sqrt(var + self.eps) * self.w[name + ".weight"] + self.w[name + ".bias"]
|
||||
|
||||
def _hidden(self, ids):
|
||||
"""ids [B, T] → final hidden states [B, T, d]."""
|
||||
w = self.w
|
||||
B, T = ids.shape
|
||||
x = w["wte.weight"][ids] + w["wpe.weight"][:T]
|
||||
mask = np.triu(np.full((T, T), -1e9, dtype=np.float32), 1)
|
||||
hd = self.d // self.n_head
|
||||
for l in range(self.n_layer):
|
||||
p = f"h.{l}."
|
||||
h = self._ln(x, p + "ln_1")
|
||||
qkv = h @ w[p + "attn.c_attn.weight"] + w[p + "attn.c_attn.bias"]
|
||||
q, k, v = np.split(qkv, 3, axis=-1)
|
||||
q = q.reshape(B, T, self.n_head, hd).transpose(0, 2, 1, 3)
|
||||
k = k.reshape(B, T, self.n_head, hd).transpose(0, 2, 3, 1)
|
||||
v = v.reshape(B, T, self.n_head, hd).transpose(0, 2, 1, 3)
|
||||
att = q @ k / math.sqrt(hd) + mask
|
||||
att = np.exp(att - att.max(-1, keepdims=True))
|
||||
att /= att.sum(-1, keepdims=True)
|
||||
a = (att @ v).transpose(0, 2, 1, 3).reshape(B, T, self.d)
|
||||
x = x + a @ w[p + "attn.c_proj.weight"] + w[p + "attn.c_proj.bias"]
|
||||
h = self._ln(x, p + "ln_2")
|
||||
h = h @ w[p + "mlp.c_fc.weight"] + w[p + "mlp.c_fc.bias"]
|
||||
h = 0.5 * h * (1 + np.tanh(math.sqrt(2 / math.pi) * (h + 0.044715 * h ** 3))) # gelu_new
|
||||
x = x + h @ w[p + "mlp.c_proj.weight"] + w[p + "mlp.c_proj.bias"]
|
||||
return self._ln(x, "ln_f")
|
||||
|
||||
def log_probs(self, texts, context=""):
|
||||
"""Natural-log probabilities of several texts after the same context. Texts of the
|
||||
same length are scored together in one batch."""
|
||||
out = [0.0] * len(texts)
|
||||
by_len = {}
|
||||
for i, t in enumerate(texts):
|
||||
if t:
|
||||
by_len.setdefault(len(t), []).append(i)
|
||||
for length, idx in by_len.items():
|
||||
ctx = self.ids(context)
|
||||
if len(ctx) + length + 1 > self.max_tokens:
|
||||
ctx = ctx[max(0, len(ctx) - (self.max_tokens - length - 1)):]
|
||||
targets = np.array([self.ids(texts[i]) for i in idx]) # [B, L]
|
||||
prefix = np.array([CLS] + ctx)
|
||||
inputs = np.concatenate([np.tile(prefix, (len(idx), 1)), targets[:, :-1]], axis=1)
|
||||
h = self._hidden(inputs)[:, len(ctx):, :] # rows predicting targets
|
||||
logits = h @ self.w["wte.weight"].T # [B, L, V]
|
||||
m = logits.max(-1, keepdims=True)
|
||||
lse = np.log(np.exp(logits - m).sum(-1)) + m[..., 0]
|
||||
picked = np.take_along_axis(logits, targets[..., None], -1)[..., 0]
|
||||
scores = (picked - lse).sum(-1)
|
||||
for j, i in enumerate(idx):
|
||||
out[i] = float(scores[j])
|
||||
return out
|
||||
|
||||
def log_prob(self, text, context=""):
|
||||
return self.log_probs([text], context)[0]
|
||||
@@ -0,0 +1,102 @@
|
||||
""""AI 選字": reranks the converter's best paths with a language model. Port of Reranker.swift.
|
||||
|
||||
score = lexicon score (log10 → ln) + λ · LM log-probability
|
||||
|
||||
Typing is never blocked for long: a rerank that runs over its time budget counts as an
|
||||
overrun, and three in a row pause AI for a minute. `system_state` lets the shell pause
|
||||
or reduce it when the machine is busy, hot or saving power.
|
||||
"""
|
||||
import math
|
||||
import time
|
||||
|
||||
OFF, SAVER, STANDARD, QUALITY = "off", "saver", "standard", "quality"
|
||||
MODES = (OFF, SAVER, STANDARD, QUALITY)
|
||||
# Paths reranked, and the time budget (seconds) of one rerank. numpy is slower than
|
||||
# the Mac's Accelerate code, so the budgets are larger than in Reranker.swift.
|
||||
K = {SAVER: 3, STANDARD: 10, QUALITY: 30}
|
||||
BUDGET = {SAVER: 0.04, STANDARD: 0.08, QUALITY: 0.2}
|
||||
|
||||
NORMAL, REDUCED, PAUSED = "normal", "reduced", "paused" # system states
|
||||
ST_OFF, ST_LOADING, ST_ACTIVE, ST_BUSY, ST_SLOW = "off", "loading", "active", "paused-busy", "paused-slow"
|
||||
|
||||
PAUSE_AFTER_OVERRUNS = 3
|
||||
SLOW_PAUSE = 60.0
|
||||
CACHE_LIMIT = 512
|
||||
|
||||
|
||||
class Reranker:
|
||||
def __init__(self, mode=STANDARD, model=None, now=time.monotonic):
|
||||
self._mode = mode
|
||||
self._model = model
|
||||
self.lam = 0.6
|
||||
self.system_state = lambda: NORMAL
|
||||
self.now = now
|
||||
self.status = ST_OFF
|
||||
self._overruns = 0
|
||||
self._paused_until = float("-inf")
|
||||
self._cache = {}
|
||||
|
||||
@property
|
||||
def mode(self):
|
||||
return self._mode
|
||||
|
||||
@mode.setter
|
||||
def mode(self, value):
|
||||
self._mode = value
|
||||
self._cache.clear()
|
||||
|
||||
@property
|
||||
def model(self):
|
||||
return self._model
|
||||
|
||||
@model.setter
|
||||
def model(self, value):
|
||||
self._model = value
|
||||
self._cache.clear()
|
||||
|
||||
def effective_mode(self):
|
||||
"""The mode used now (after the system state), or None when AI is not used."""
|
||||
if self._mode == OFF:
|
||||
self.status = ST_OFF
|
||||
return None
|
||||
if self._model is None:
|
||||
self.status = ST_LOADING
|
||||
return None
|
||||
if self.now() < self._paused_until:
|
||||
self.status = ST_SLOW
|
||||
return None
|
||||
state = self.system_state()
|
||||
if state == PAUSED:
|
||||
self.status = ST_BUSY
|
||||
return None
|
||||
self.status = ST_ACTIVE
|
||||
return SAVER if state == REDUCED else self._mode
|
||||
|
||||
def choose(self, converter, syllables, fixed, context):
|
||||
"""The reranked conversion [((start, end), text)], or None for the plain result."""
|
||||
mode = self.effective_mode()
|
||||
if mode is None or not syllables:
|
||||
return None
|
||||
key = (mode, context, tuple(s.reading for s in syllables), tuple(sorted(fixed.items())))
|
||||
if key in self._cache:
|
||||
return self._cache[key]
|
||||
paths = converter.kbest(syllables, K[mode], fixed)
|
||||
if len(paths) <= 1:
|
||||
return paths[0][1] if paths else None
|
||||
start = self.now()
|
||||
lm = self._model.log_probs(["".join(t for _, t in segs) for _, segs in paths], context)
|
||||
best = max(range(len(paths)), key=lambda i: paths[i][0] * math.log(10) + self.lam * lm[i])
|
||||
if self.now() - start > BUDGET[mode]:
|
||||
# Too slow: use this result (it is computed) but don't cache it, and pause AI
|
||||
# after a few overruns in a row.
|
||||
self._overruns += 1
|
||||
if self._overruns >= PAUSE_AFTER_OVERRUNS:
|
||||
self._paused_until = self.now() + SLOW_PAUSE
|
||||
self._overruns = 0
|
||||
self.status = ST_SLOW
|
||||
return paths[best][1]
|
||||
self._overruns = 0
|
||||
if len(self._cache) >= CACHE_LIMIT:
|
||||
self._cache.clear()
|
||||
self._cache[key] = paths[best][1]
|
||||
return paths[best][1]
|
||||
@@ -0,0 +1,95 @@
|
||||
"""Decides when "AI 選字" should step back (Windows and Linux; the Mac app has its own).
|
||||
|
||||
paused — CPU busy (over 90%), or the machine is very hot (Linux thermal zones ≥ 95 °C)
|
||||
reduced — power saving (Windows battery saver; Linux low-power profile or battery ≤ 20%)
|
||||
normal — otherwise. Checked at most every 2 seconds.
|
||||
"""
|
||||
import glob
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
from .reranker import NORMAL, PAUSED, REDUCED
|
||||
|
||||
|
||||
class SystemMonitor:
|
||||
interval = 2.0
|
||||
busy_load = 0.9
|
||||
|
||||
def __init__(self, now=time.monotonic):
|
||||
self.now = now
|
||||
self._cached = (float("-inf"), NORMAL)
|
||||
self._cpu_prev = None
|
||||
|
||||
def state(self):
|
||||
t, s = self._cached
|
||||
if self.now() - t < self.interval:
|
||||
return s
|
||||
try:
|
||||
s = self._compute()
|
||||
except Exception:
|
||||
s = NORMAL
|
||||
self._cached = (self.now(), s)
|
||||
return s
|
||||
|
||||
def _compute(self):
|
||||
if sys.platform == "win32":
|
||||
return self._windows()
|
||||
return self._linux()
|
||||
|
||||
# --- Linux ---
|
||||
|
||||
def _linux(self):
|
||||
if os.getloadavg()[0] / max(1, os.cpu_count() or 1) > self.busy_load:
|
||||
return PAUSED
|
||||
temps = []
|
||||
for path in glob.glob("/sys/class/thermal/thermal_zone*/temp"):
|
||||
try:
|
||||
temps.append(int(open(path).read().strip()) / 1000)
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
if temps and max(temps) >= 95:
|
||||
return PAUSED
|
||||
try:
|
||||
if open("/sys/firmware/acpi/platform_profile").read().strip() in ("low-power", "quiet"):
|
||||
return REDUCED
|
||||
except OSError:
|
||||
pass
|
||||
for bat in glob.glob("/sys/class/power_supply/BAT*"):
|
||||
try:
|
||||
status = open(bat + "/status").read().strip()
|
||||
capacity = int(open(bat + "/capacity").read().strip())
|
||||
except (OSError, ValueError):
|
||||
continue
|
||||
if status == "Discharging" and capacity <= 20:
|
||||
return REDUCED
|
||||
return NORMAL
|
||||
|
||||
# --- Windows ---
|
||||
|
||||
def _windows(self):
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
|
||||
class FILETIME(ctypes.Structure):
|
||||
_fields_ = [("lo", wintypes.DWORD), ("hi", wintypes.DWORD)]
|
||||
|
||||
idle, kernel, user = FILETIME(), FILETIME(), FILETIME()
|
||||
ctypes.windll.kernel32.GetSystemTimes(ctypes.byref(idle), ctypes.byref(kernel), ctypes.byref(user))
|
||||
values = [(f.hi << 32) | f.lo for f in (idle, kernel, user)]
|
||||
prev, self._cpu_prev = self._cpu_prev, values
|
||||
if prev:
|
||||
d_idle, d_kernel, d_user = (a - b for a, b in zip(values, prev))
|
||||
total = d_kernel + d_user # kernel time includes idle time
|
||||
if total > 0 and 1 - d_idle / total > self.busy_load:
|
||||
return PAUSED
|
||||
|
||||
class SYSTEM_POWER_STATUS(ctypes.Structure):
|
||||
_fields_ = [("ACLineStatus", ctypes.c_ubyte), ("BatteryFlag", ctypes.c_ubyte),
|
||||
("BatteryLifePercent", ctypes.c_ubyte), ("SystemStatusFlag", ctypes.c_ubyte),
|
||||
("BatteryLifeTime", wintypes.DWORD), ("BatteryFullLifeTime", wintypes.DWORD)]
|
||||
|
||||
status = SYSTEM_POWER_STATUS()
|
||||
if ctypes.windll.kernel32.GetSystemPowerStatus(ctypes.byref(status)) and status.SystemStatusFlag == 1:
|
||||
return REDUCED # battery saver on
|
||||
return NORMAL
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.4 KiB |
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "自動注音 AutoZhuyin",
|
||||
"version": "0.11.0",
|
||||
"version": "0.13.2",
|
||||
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
||||
"locale": "zh-Hant-TW",
|
||||
"fallbackLocale": "zh-TW",
|
||||
|
||||
@@ -31,7 +31,7 @@
|
||||
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字;
|
||||
選過的字會被記住,下次優先出現
|
||||
Esc 取消組字
|
||||
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線),
|
||||
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線,工作列圖示變成 A),
|
||||
再單按一次回到中英自動判斷,會出現「英文」/「中英自動」小提示;
|
||||
打到一半時按,同 Tab 整段翻轉
|
||||
Shift+字母 打大寫字母(也會強制英文)
|
||||
@@ -43,6 +43,15 @@
|
||||
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
|
||||
|
||||
|
||||
AI 選字
|
||||
-------
|
||||
用中研院 CKIP 的繁體中文小型語言模型,看前後文挑同音字(他跑「得」很快、請你「再」說一次),
|
||||
完全在本機運算,不連網。在工作列的輸入法圖示上按右鍵(或語言列的設定按鈕)可以選:
|
||||
AI 選字:關閉/省電/標準(預設)/高品質
|
||||
系統忙碌時自動暫停 AI(預設開啟:CPU 很忙時暫停,省電模式時改用省電)
|
||||
選單裡會顯示目前狀態。電腦比較慢時建議用「省電」,運算太慢時也會自動暫停一分鐘。
|
||||
|
||||
|
||||
回報問題
|
||||
--------
|
||||
請告訴我:
|
||||
|
||||
@@ -423,5 +423,105 @@ class UserHistoryTests(unittest.TestCase):
|
||||
self.assertEqual(set(h.data["ㄊㄧㄢ"]), {"添", "田"})
|
||||
|
||||
|
||||
class FakeLM:
|
||||
"""Likes one string; `on_call` can advance a fake clock."""
|
||||
def __init__(self, liked):
|
||||
self.liked, self.calls, self.on_call = liked, 0, lambda: None
|
||||
|
||||
def log_probs(self, texts, context=""):
|
||||
self.calls += len(texts)
|
||||
self.on_call()
|
||||
return [0.0 if self.liked in t else -50.0 for t in texts]
|
||||
|
||||
|
||||
class RerankerTests(unittest.TestCase):
|
||||
def engine(self, reranker):
|
||||
e = make_engine()
|
||||
e.reranker = reranker
|
||||
return e
|
||||
|
||||
def test_rerank_picks_what_the_model_likes(self):
|
||||
from autozhuyin.core.reranker import Reranker, ST_ACTIVE
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(None)), "天")
|
||||
r = Reranker("standard", FakeLM("添"))
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "添")
|
||||
self.assertEqual(r.status, ST_ACTIVE)
|
||||
|
||||
def test_off_and_loading_use_the_lexicon(self):
|
||||
from autozhuyin.core.reranker import Reranker, ST_LOADING, ST_OFF
|
||||
off = Reranker("off", FakeLM("添"))
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(off)), "天")
|
||||
self.assertEqual(off.status, ST_OFF)
|
||||
loading = Reranker("standard", None)
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(loading)), "天")
|
||||
self.assertEqual(loading.status, ST_LOADING)
|
||||
|
||||
def test_paused_and_reduced(self):
|
||||
from autozhuyin.core.reranker import PAUSED, REDUCED, Reranker, ST_BUSY
|
||||
r = Reranker("standard", FakeLM("添"))
|
||||
r.system_state = lambda: PAUSED
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "天")
|
||||
self.assertEqual(r.status, ST_BUSY)
|
||||
lm = FakeLM("添")
|
||||
r = Reranker("quality", lm)
|
||||
r.system_state = lambda: REDUCED
|
||||
type_keys("wu0 \n", self.engine(r))
|
||||
self.assertLessEqual(lm.calls, 3)
|
||||
|
||||
def test_slow_model_is_paused(self):
|
||||
from autozhuyin.core.reranker import Reranker, ST_SLOW
|
||||
clock = [0.0]
|
||||
lm = FakeLM("添")
|
||||
lm.on_call = lambda: clock.__setitem__(0, clock[0] + 1) # every batch takes a "second"
|
||||
r = Reranker("standard", lm, now=lambda: clock[0])
|
||||
for _ in range(3):
|
||||
type_keys("wu0 \n", self.engine(r))
|
||||
self.assertEqual(r.status, ST_SLOW)
|
||||
lm.on_call = lambda: None
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "天") # paused
|
||||
clock[0] += 61
|
||||
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "添") # back after a minute
|
||||
|
||||
def test_kbest_first_is_viterbi(self):
|
||||
syl = [s.syllable for s in bopomofo.parse(list("rup wu0 wu0 fu4")).syllables]
|
||||
conv = Converter(LEX)
|
||||
best = conv.kbest(syl, 5)
|
||||
self.assertEqual("".join(t for _, t in best[0][1]), "".join(t for _, t in conv.convert(syl)))
|
||||
self.assertEqual(len(best), 5)
|
||||
|
||||
|
||||
class RealModelTests(unittest.TestCase):
|
||||
"""Needs numpy and data/lm.bin (scripts/fetch-lm.sh, tools/build_data.py); skipped otherwise."""
|
||||
def test_fixes_homophones(self):
|
||||
from autozhuyin.core import lm
|
||||
from autozhuyin.core.reranker import Reranker
|
||||
path = os.path.join(DATA, "lm.bin")
|
||||
if not lm.available() or not os.path.exists(path):
|
||||
self.skipTest("needs numpy and the model")
|
||||
model = lm.GPT2LanguageModel(path, os.path.join(DATA, "lm-vocab.txt"))
|
||||
self.assertAlmostEqual(model.log_prob("他在家"), -14.329, delta=0.05)
|
||||
cases = {"w8 ql32k71u3ji3dj94cp32ji ": "他跑得比我快很多", "fu/3su3y94gji u h4cl3a87": "請你再說一次好嗎"}
|
||||
for keys, want in cases.items():
|
||||
self.assertNotEqual(type_keys(keys + "\n", make_engine()), want)
|
||||
e = make_engine()
|
||||
e.reranker = Reranker("standard", model)
|
||||
self.assertEqual(type_keys(keys + "\n", e), want)
|
||||
|
||||
|
||||
class AISettingsTests(unittest.TestCase):
|
||||
def test_settings_round_trip_and_status(self):
|
||||
import tempfile
|
||||
from autozhuyin.core.ai import AI
|
||||
user = tempfile.mkdtemp()
|
||||
ai = AI(tempfile.mkdtemp(), user, background=False) # no model files here
|
||||
self.assertTrue(ai.error) # "需要 numpy" or "AI 模型未安裝"
|
||||
self.assertEqual(ai.status_text(), ai.error)
|
||||
ai.set_mode("saver")
|
||||
ai.set_auto_pause(False)
|
||||
again = AI(tempfile.mkdtemp(), user, background=False)
|
||||
self.assertEqual(again.reranker.mode, "saver")
|
||||
self.assertFalse(again.auto_pause)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -40,6 +40,9 @@ class FakeTextService:
|
||||
def setCandidateCursor(self, p): self.reply["candidateCursor"] = p
|
||||
|
||||
def showMessage(self, message, duration=3): self.reply["showMessage"] = message
|
||||
def addButton(self, button_id, **kw): self.reply.setdefault("addButton", []).append(button_id)
|
||||
def removeButton(self, button_id): self.reply.setdefault("removeButton", []).append(button_id)
|
||||
def changeButton(self, button_id, **kw): self.reply.setdefault("changeButton", []).append((button_id, kw))
|
||||
|
||||
def setShowCandidates(self, show):
|
||||
self.showCandidates = show
|
||||
@@ -47,6 +50,7 @@ class FakeTextService:
|
||||
|
||||
|
||||
textService.TextService = FakeTextService
|
||||
textService.COMMAND_LEFT_CLICK, textService.COMMAND_RIGHT_CLICK, textService.COMMAND_MENU = 0, 1, 2
|
||||
sys.modules["textService"] = textService
|
||||
|
||||
from autozhuyin.autozhuyin_ime import AutoZhuyinTextService, _shared_classifier # noqa: E402
|
||||
@@ -114,6 +118,29 @@ class WrapperTests(unittest.TestCase):
|
||||
self.assertEqual(type_text(self.svc, "`<"), ",")
|
||||
self.assertEqual(type_text(self.svc, "ji3`."), "我。")
|
||||
|
||||
def test_settings_menu(self):
|
||||
from autozhuyin import autozhuyin_ime as ime
|
||||
self.assertIn("autozhuyin-settings", self.svc.reply["addButton"])
|
||||
menu = self.svc.onMenu("autozhuyin-settings")
|
||||
texts = [m.get("text") for m in menu]
|
||||
self.assertIn("AI 選字:標準", texts)
|
||||
self.assertIn("清除學習紀錄", texts)
|
||||
self.svc.onCommand(ime.ID_AI_MODE["saver"], 2)
|
||||
checked = [m["text"] for m in self.svc.onMenu("autozhuyin-settings") if m.get("checked")]
|
||||
self.assertIn("AI 選字:省電", checked)
|
||||
self.svc.onCommand(ime.ID_AI_MODE["standard"], 2)
|
||||
self.assertIsNone(self.svc.onMenu("other"))
|
||||
|
||||
def test_mode_icon_click_toggles_english(self):
|
||||
self.svc.reply = {}
|
||||
self.svc.onCommand(1, 0)
|
||||
self.assertTrue(self.svc.engine.english_mode)
|
||||
(button, kw), = self.svc.reply["changeButton"]
|
||||
self.assertEqual(button, "windows-mode-icon")
|
||||
self.assertTrue(kw["icon"].endswith("icon-en.ico") or kw["icon"].endswith("icon.ico"))
|
||||
self.svc.onCommand(1, 0)
|
||||
self.assertFalse(self.svc.engine.english_mode)
|
||||
|
||||
def test_composition_shown(self):
|
||||
type_text(self.svc, "su3c")
|
||||
self.assertEqual(self.svc.reply["compositionString"], "你ㄏ")
|
||||
@@ -152,6 +179,12 @@ class ShiftTapWrapperTests(unittest.TestCase):
|
||||
self.svc = AutoZhuyinTextService(client=None)
|
||||
self.svc.onActivate()
|
||||
|
||||
def test_shift_tap_changes_icon(self):
|
||||
tap_shift(self.svc)
|
||||
self.assertEqual(self.svc.reply["changeButton"][0][0], "windows-mode-icon")
|
||||
tap_shift(self.svc)
|
||||
self.assertIn("changeButton", self.svc.reply)
|
||||
|
||||
def test_tap_forces_english_lowercase(self):
|
||||
reply = tap_shift(self.svc)
|
||||
self.assertEqual(reply.get("showMessage"), "英文")
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
- english-dict.txt: filtered from a system word list (default /usr/share/dict/words,
|
||||
Webster's 2nd edition, public domain), since Windows has no such file
|
||||
- english-bigram.json: precomputed letter-bigram table
|
||||
- lm.bin, lm-vocab.txt: the "AI 選字" language model, copied from build/lm/ if present
|
||||
|
||||
Usage: build_data.py [word-list]
|
||||
"""
|
||||
@@ -34,4 +35,10 @@ common = {w.strip().lower() for w in (data / "english-common.txt").read_text(enc
|
||||
bigram = train_bigram(sorted(common | set(words)), common)
|
||||
(data / "english-bigram.json").write_text(json.dumps(bigram), encoding="utf-8")
|
||||
|
||||
# "AI 選字" language model (scripts/fetch-lm.sh makes build/lm/); optional.
|
||||
lm_dir = ROOT / "build" / "lm"
|
||||
for name in ("lm.bin", "lm-vocab.txt"):
|
||||
if (lm_dir / name).exists():
|
||||
shutil.copy(lm_dir / name, data / name)
|
||||
|
||||
print("data ready: %d dictionary words, %d bigrams" % (len(words), len(bigram["table"])))
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Downloads numpy for one Python version / platform from PyPI and unpacks it into
|
||||
<dest>/<python tag>/ (numpy, numpy.libs, dist-info; test folders are dropped).
|
||||
|
||||
fetch_numpy.py <dest> <python tag> <platform>
|
||||
fetch_numpy.py autozhuyin/vendor cp38 win32
|
||||
fetch_numpy.py ~/.local/share/autozhuyin/vendor auto auto (this Python, Linux)
|
||||
|
||||
The newest numpy release that has a wheel for that combination is used, and its
|
||||
SHA-256 is checked against PyPI. Standard library only.
|
||||
"""
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import platform
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def version_key(v):
|
||||
return tuple(int(x) for x in re.findall(r"\d+", v))
|
||||
|
||||
|
||||
def linux_platforms():
|
||||
machine = platform.machine().lower()
|
||||
arch = {"amd64": "x86_64", "arm64": "aarch64"}.get(machine, machine)
|
||||
return [f"manylinux_2_17_{arch}", f"manylinux2014_{arch}", f"manylinux_2_27_{arch}", f"manylinux_2_28_{arch}"]
|
||||
|
||||
|
||||
def find_wheel(tag, platforms):
|
||||
with urllib.request.urlopen("https://pypi.org/pypi/numpy/json", timeout=60) as r:
|
||||
releases = json.load(r)["releases"]
|
||||
for version in sorted(releases, key=version_key, reverse=True):
|
||||
if re.search(r"[a-z]", version): # skip pre-releases (rc, b, a, dev)
|
||||
continue
|
||||
for f in releases[version]:
|
||||
name = f["filename"]
|
||||
if not name.endswith(".whl") or f.get("yanked"):
|
||||
continue
|
||||
if f"-{tag}-{tag}-" not in name: # also skips free-threaded "cp315t" builds
|
||||
continue
|
||||
plat = name[:-4].split("-")[-1]
|
||||
if any(p in plat.split(".") for p in platforms):
|
||||
return version, f
|
||||
return None, None
|
||||
|
||||
|
||||
def main(dest, tag, plat):
|
||||
if tag == "auto":
|
||||
tag = "cp%d%d" % sys.version_info[:2]
|
||||
platforms = linux_platforms() if plat == "auto" else [plat]
|
||||
version, f = find_wheel(tag, platforms)
|
||||
if not f:
|
||||
sys.exit(f"no numpy wheel for {tag} {platforms}")
|
||||
print(f"numpy {version} for {tag}: {f['filename']} ({f['size'] / 1e6:.1f} MB)")
|
||||
with urllib.request.urlopen(f["url"], timeout=300) as r:
|
||||
data = r.read()
|
||||
if hashlib.sha256(data).hexdigest() != f["digests"]["sha256"]:
|
||||
sys.exit("checksum mismatch")
|
||||
out = Path(dest).expanduser() / tag
|
||||
if out.exists():
|
||||
shutil.rmtree(out)
|
||||
out.mkdir(parents=True)
|
||||
with zipfile.ZipFile(io.BytesIO(data)) as z:
|
||||
for info in z.infolist():
|
||||
parts = Path(info.filename).parts
|
||||
if "tests" in parts or parts[0].endswith(".data"):
|
||||
continue
|
||||
z.extract(info, out)
|
||||
print(f"unpacked to {out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) != 4:
|
||||
sys.exit(__doc__)
|
||||
main(*sys.argv[1:])
|
||||
Reference in New Issue
Block a user