Compare commits

..
8 Commits
Author SHA1 Message Date
ays953 cfd095b3b0 Release v0.13.2 2026-11-13 17:08:09 +08:00
ays953andClaude Opus 5.5 f323a29552 Don't open Keyboard settings on every Shift tap
InputModes.select called TISEnableInputSource before every switch, and macOS
opens System Settings > Keyboard whenever a source is enabled, even one that is
already enabled. Enable only when the mode is not enabled yet (also in
--register).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 17:07:49 +08:00
ays953 aa00b793ce Release v0.13.1 2026-11-13 17:03:26 +08:00
ays953andClaude Opus 5.5 b3b7589379 Name the input modes in System Settings
The modes showed their raw IDs (com.autozhuyin.inputmethod.AutoZhuyin.Auto).
InfoPlist.strings (zh-Hant, en, Base) now name them 自動注音(中英自動) and
自動注音(英文); build.sh copies the .lproj folders into the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 17:03:07 +08:00
ays953 c440cd7177 Release v0.13.0 2026-11-13 16:56:27 +08:00
ays953andClaude Opus 5.5 6443f771a0 Show English mode in the input method icon
macOS: the input method now has two input modes in Info.plist, 中英自動 ("注")
and 英文 ("A"), each with its own menu-bar icon. A Shift tap with nothing typed
switches the mode (TISSelectInputSource), and choosing a mode in the menu bar
switches the engine (setValue:forTag:). --register enables both modes.
Windows: the taskbar IME icon switches between icon.ico and icon-en.ico.
Linux: the IBus mode property also switches its icon.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 16:56:08 +08:00
ays953 0155ddd16e Release v0.12.0 2026-11-13 16:52:01 +08:00
ays953andClaude Opus 5.5 c8b3c4ae6f AI candidate selection on Windows and Linux
Python port of the reranker: core/lm.py runs the same CKIP gpt2-tiny-chinese
model with numpy (batched per candidate length, scores match the Swift and
reference implementations), Converter.kbest, core/reranker.py (modes, time
budget, pause after overruns, cache), core/system_monitor.py (Windows: CPU usage
and battery saver; Linux: load average, thermal zones, low-power profile and low
battery) and core/ai.py (settings.json, background model loading, status line).

Windows: numpy wheels for PIME's 32-bit Python 3.8 (PIME 1.3) and 3.15 (PIME 2)
are bundled under autozhuyin/vendor and picked by Python version; a settings menu
on the taskbar IME icon and the language bar (left click on the icon toggles
English mode). Linux: install.sh uses the system numpy or downloads the matching
wheel from PyPI (tools/fetch_numpy.py, SHA-256 checked); an IBus settings menu.
Without numpy or the model the input method works as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 16:51:41 +08:00
33 changed files with 1112 additions and 30 deletions
+23 -10
View File
@@ -12,12 +12,12 @@
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷;英文模式時輸入法圖示會從「注」變成「A」
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
- **記住選字**:從候選字選過的字詞會被記住,下次打同樣的讀音會優先出現(選越多次越優先)
- **AI 選字(macOS)**:用中研院 CKIP 的繁中小型語言模型看前後文挑同音字(他跑「得」很快、請你「再」說一次),完全在本機運算;可調整或關閉,系統忙碌時自動暫停
- **AI 選字**:用中研院 CKIP 的繁中小型語言模型看前後文挑同音字(他跑「得」很快、請你「再」說一次),完全在本機運算;可調整或關閉,系統忙碌時自動暫停
- **中間補字、刪字**:游標移到前面後直接打注音,會插在游標處;Backspace 刪掉游標前的那個字
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`;任何時候按 `` ` `` 再按標點鍵,一定得到全形標點(例如 `` `< `` = `,`)
- **完整詞庫**:約 17 萬筆,來自小麥注音 McBopomofo(MIT 授權)
@@ -34,6 +34,8 @@
2. 系統設定 → 鍵盤 → 文字輸入 → 編輯… → + → 繁體中文 → **自動注音**。
3. 清單裡沒出現的話,登出再登入一次。
從 0.12 以前的版本更新:輸入法現在分成「中英自動」和「英文」兩個模式(各有圖示),安裝時會自動加入。如果輸入法清單裡還留著舊的「自動注音」項目,把它移除即可。
PKG 會裝到 `/Library/Input Methods/`,系統上所有使用者都能用。
### Windows
@@ -67,7 +69,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
| ← → | 在組字區移動游標(最左停在第一個中文字後面)。游標在中間時:直接打注音會**插入**新字,Backspace 刪掉游標前的字,↓ 選游標前的字 |
| ↓ | 開啟游標前那個字或詞的候選字(游標預設在最後),數字鍵 1–9 選字,Esc 關閉 |
| Esc | 取消組字 |
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷。打到一半時:同 Tab,整段翻轉 |
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷,圖示會變成「A」(Mac 的選單列、Windows 的工作列、Linux 的 IBus 狀態)。Mac 也可以直接在選單列選「英文」模式。打到一半時:同 Tab,整段翻轉 |
| Shift + 字母 | 打大寫字母(也會強制英文) |
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
@@ -97,7 +99,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
轉換用 Viterbi 演算法,在詞庫中找出整串音節機率最高的詞組組合。
### AI 選字(macOS)
### AI 選字
詞庫只知道每個詞本身有多常見,不看前後文。AI 選字讓語言模型當裁判:
@@ -105,7 +107,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
2. 模型 [CKIP GPT2-tiny](https://huggingface.co/ckiplab/gpt2-tiny-chinese)(中研院詞庫小組,繁體中文,約 1200 萬參數,22 MB)計算每個組合出現的機率,並參考剛送出的前文。
3. 分數 = 詞庫分數 + 0.6 × 模型分數,取最高的。選過的候選字仍然固定不變。
輸入法選單可以調整:
輸入法選單可以調整(Mac:選單列的輸入法選單;Windows:工作列輸入法圖示按右鍵或語言列的設定按鈕;Linux:IBus 選單的「設定」):
| 模式 | 比較幾個組合 | 每次運算上限 |
|---|---|---|
@@ -116,7 +118,13 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
不會拖慢打字的機制:
- 每次運算超過上限就直接放棄,改用詞庫結果;連續 3 次超時就暫停一分鐘。
- 「系統忙碌時自動暫停 AI」(預設開啟):Mac 過熱,或 CPU 負載(每核心 1 分鐘平均)超過 0.9 時暫停;低耗電模式下降到省電模式。選單最下面會顯示目前狀態。
- 「系統忙碌時自動暫停 AI」(預設開啟),選單裡會顯示目前狀態:
| | 暫停 AI | 降到省電模式 |
|---|---|---|
| macOS | 過熱,或 CPU 負載(每核心 1 分鐘平均)超過 0.9 | 低耗電模式 |
| Windows | CPU 使用率超過 90% | 省電模式(電池保護) |
| Linux | CPU 負載超過 0.9,或溫度達 95 °C | 低耗電設定檔,或用電池且電量 ≤ 20% |
- 模型在背景載入,載入完成前照常用詞庫。
實測(`training/lm/eval_rerank.py`,標準模式):
@@ -126,7 +134,12 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
| 日常對話 60 句 | 字正確率 96.6%,整句全對 73% | 97.5%,80% |
| 維基百科 714 句 | 92.9%,31% | 94.7%,44% |
錯字大約少了 25–30%。Windows、Linux 版目前還沒有 AI 選字。
錯字大約少了 25–30%。
Windows、Linux 版用 numpy 執行同一個模型(`windows/autozhuyin/core/lm.py`),所以時間上限放寬為省電 40、標準 80、高品質 200 毫秒,超時的那次結果照用但不快取,連續 3 次超時暫停一分鐘。
- Windows:安裝包附上 PIME 兩種 Python(1.3 版的 3.8、2.x 版的 3.15,都是 32 位元)用的 numpy,安裝檔約 40 MB。
- Linux:系統沒有 numpy 時,`install.sh` 會從 PyPI 下載符合的版本到 `~/.local/share/autozhuyin/vendor/`。
- 設定存在 `%APPDATA%\PIME\autozhuyin\settings.json`、`~/.config/autozhuyin/settings.json`。
## 個人資料
@@ -311,8 +324,7 @@ THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝
- [x] 用資料訓練判斷權重(合成資料)
- [ ] 加入真實打字資料一起訓練
- [ ] 已送出的英文單字若判斷錯誤,可以「事後翻轉」
- [x] AI 選字:語言模型參考前後文挑同音字(macOS)
- [ ] AI 選字移植到 Windows、Linux
- [x] AI 選字:語言模型參考前後文挑同音字(macOS、Windows、Linux)
- [x] 使用者詞頻學習(選過的候選字優先)
- [ ] 偏好設定視窗
@@ -321,6 +333,7 @@ THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝
- 詞庫資料:[小麥注音 McBopomofo](https://github.com/openvanilla/McBopomofo)(MIT License)
- Windows 輸入法框架:[PIME](https://github.com/EasyIME/PIME)(LGPL 2.1,使用者自行安裝,本專案不包含其程式碼)
- 英文字典:Webster's Second International Dictionary(公有領域,macOS 的 `/usr/share/dict/words`)
- AI 選字語言模型:[CKIP GPT2 Tiny Chinese](https://github.com/ckiplab/ckip-transformers)(中研院詞庫小組,GPL-3.0,全文在 `licenses/GPL-3.0.txt`,Mac 版一併放在 app 的 Resources)
- AI 選字語言模型:[CKIP GPT2 Tiny Chinese](https://github.com/ckiplab/ckip-transformers)(中研院詞庫小組,GPL-3.0,全文在 `licenses/GPL-3.0.txt`,並一起打包進三個平台的安裝檔)
- 數值運算:[NumPy](https://numpy.org)(BSD 3-Clause,Windows 版附上、Linux 版安裝時下載)
完整的第三方授權聲明(含小麥注音 MIT 授權全文)見 [THIRD_PARTY_LICENSES.txt](THIRD_PARTY_LICENSES.txt),Mac 版放在 `AutoZhuyin.app/Contents/Resources/`,Windows 版放在 zip 根目錄和安裝後的 `autozhuyin` 資料夾。
+6
View File
@@ -0,0 +1,6 @@
/* Names shown in System Settings and the input menu. */
CFBundleName = "自動注音";
CFBundleDisplayName = "自動注音";
"com.autozhuyin.inputmethod.AutoZhuyin" = "自動注音";
"com.autozhuyin.inputmethod.AutoZhuyin.Auto" = "自動注音(中英自動)";
"com.autozhuyin.inputmethod.AutoZhuyin.English" = "自動注音(英文)";
+63 -2
View File
@@ -15,9 +15,70 @@
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>0.11.0</string>
<string>0.13.2</string>
<key>CFBundleVersion</key>
<string>19</string>
<string>23</string>
<key>ComponentInputModeDict</key>
<dict>
<key>tsInputModeListKey</key>
<dict>
<key>com.autozhuyin.inputmethod.AutoZhuyin.Auto</key>
<dict>
<key>TISInputSourceID</key>
<string>com.autozhuyin.inputmethod.AutoZhuyin.Auto</string>
<key>TISIntendedLanguage</key>
<string>zh-Hant</string>
<key>tsInputModeAlternateMenuIconFileKey</key>
<string>icon.tiff</string>
<key>tsInputModeCharacterRepertoireKey</key>
<array>
<string>Hant</string>
</array>
<key>tsInputModeDefaultStateKey</key>
<true/>
<key>tsInputModeIsVisibleKey</key>
<true/>
<key>tsInputModeMenuIconFileKey</key>
<string>icon.tiff</string>
<key>tsInputModePaletteIconFileKey</key>
<string>icon.tiff</string>
<key>tsInputModePrimaryInScriptKey</key>
<true/>
<key>tsInputModeScriptKey</key>
<string>smTradChinese</string>
</dict>
<key>com.autozhuyin.inputmethod.AutoZhuyin.English</key>
<dict>
<key>TISInputSourceID</key>
<string>com.autozhuyin.inputmethod.AutoZhuyin.English</string>
<key>TISIntendedLanguage</key>
<string>zh-Hant</string>
<key>tsInputModeAlternateMenuIconFileKey</key>
<string>icon-en.tiff</string>
<key>tsInputModeCharacterRepertoireKey</key>
<array>
<string>Hant</string>
</array>
<key>tsInputModeDefaultStateKey</key>
<true/>
<key>tsInputModeIsVisibleKey</key>
<true/>
<key>tsInputModeMenuIconFileKey</key>
<string>icon-en.tiff</string>
<key>tsInputModePaletteIconFileKey</key>
<string>icon-en.tiff</string>
<key>tsInputModePrimaryInScriptKey</key>
<false/>
<key>tsInputModeScriptKey</key>
<string>smTradChinese</string>
</dict>
</dict>
<key>tsVisibleInputModeOrderedArrayKey</key>
<array>
<string>com.autozhuyin.inputmethod.AutoZhuyin.Auto</string>
<string>com.autozhuyin.inputmethod.AutoZhuyin.English</string>
</array>
</dict>
<key>InputMethodConnectionName</key>
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
<key>InputMethodServerControllerClass</key>
+6
View File
@@ -0,0 +1,6 @@
/* Names shown in System Settings and the input menu. */
CFBundleName = "AutoZhuyin";
CFBundleDisplayName = "AutoZhuyin";
"com.autozhuyin.inputmethod.AutoZhuyin" = "AutoZhuyin";
"com.autozhuyin.inputmethod.AutoZhuyin.Auto" = "AutoZhuyin (Auto)";
"com.autozhuyin.inputmethod.AutoZhuyin.English" = "AutoZhuyin (English)";
@@ -0,0 +1,6 @@
/* Names shown in System Settings and the input menu. */
CFBundleName = "自動注音";
CFBundleDisplayName = "自動注音";
"com.autozhuyin.inputmethod.AutoZhuyin" = "自動注音";
"com.autozhuyin.inputmethod.AutoZhuyin.Auto" = "自動注音(中英自動)";
"com.autozhuyin.inputmethod.AutoZhuyin.English" = "自動注音(英文)";
+16
View File
@@ -1,3 +1,4 @@
import Carbon
import Cocoa
import InputMethodKit
import AutoZhuyinCore
@@ -74,8 +75,11 @@ final class InputController: IMKInputController {
otherModifiers: !flags.intersection([.command, .control, .option, .function]).isEmpty,
time: event.timestamp)
if tapped {
let idle = engine.keys.isEmpty
_ = engine.handle(.shiftTap)
updateComposition(client)
// English mode on/off: switch input mode so the menu-bar icon shows it.
if idle { InputModes.select(english: engine.isEnglishMode) }
}
return false
}
@@ -152,6 +156,18 @@ final class InputController: IMKInputController {
}
}
/// The input mode changed (menu bar, keyboard shortcut, or our own Shift tap).
override func setValue(_ value: Any!, forTag tag: Int, client sender: Any!) {
if tag == kTextServiceInputModePropertyTag, let id = value as? String {
let english = InputModes.isEnglish(id)
if english != engine.isEnglishMode {
commitComposition(sender)
engine.setEnglishMode(english)
}
}
super.setValue(value, forTag: tag, client: sender)
}
override func commitComposition(_ sender: Any!) {
guard let client = sender as? IMKTextInput else { return }
commit(engine.commitAll(), to: client)
+39
View File
@@ -0,0 +1,39 @@
import Carbon
import Foundation
/// The two input modes declared in Info.plist (ComponentInputModeDict). Each has its own
/// menu-bar icon, so switching modes is how the icon shows "中英自動" vs "英文".
enum InputModes {
static let auto = "com.autozhuyin.inputmethod.AutoZhuyin.Auto"
static let english = "com.autozhuyin.inputmethod.AutoZhuyin.English"
static func isEnglish(_ modeID: String) -> Bool { modeID == english }
private static func source(_ id: String) -> TISInputSource? {
let filter = [kTISPropertyInputSourceID as String: id] as CFDictionary
guard let list = TISCreateInputSourceList(filter, true)?.takeRetainedValue() as? [TISInputSource] else { return nil }
return list.first
}
private static func isEnabled(_ src: TISInputSource) -> Bool {
guard let p = TISGetInputSourceProperty(src, kTISPropertyInputSourceIsEnabled) else { return false }
return CFBooleanGetValue(Unmanaged<CFBoolean>.fromOpaque(p).takeUnretainedValue())
}
/// Makes the given mode current. Returns false on failure.
/// Only enables the mode if it is not enabled yet: TISEnableInputSource opens System
/// Settings → Keyboard, even for a source that is already enabled.
@discardableResult
static func select(english: Bool) -> Bool {
guard let src = source(english ? InputModes.english : InputModes.auto) else { return false }
if !isEnabled(src) { TISEnableInputSource(src) }
return TISSelectInputSource(src) == noErr
}
/// Called by `--register`: enable both modes (if not yet) so they are in the input menu.
static func enableAll() {
for id in [auto, english] {
if let src = source(id), !isEnabled(src) { TISEnableInputSource(src) }
}
}
}
+1
View File
@@ -6,6 +6,7 @@ import InputMethodKit
// in System Settings without logging out (used by the installer's postinstall script).
if CommandLine.arguments.contains("--register") {
let status = TISRegisterInputSource(Bundle.main.bundleURL as CFURL)
if status == noErr { InputModes.enableAll() } // "中英自動" and "英文" (menu-bar icons)
print(status == noErr ? "Registered \(Bundle.main.bundlePath)" : "TISRegisterInputSource failed: \(status)")
exit(status == noErr ? 0 : 1)
}
@@ -417,6 +417,16 @@ public final class CompositionEngine {
}
}
/// Turns English mode on or off directly (e.g. the input mode was switched in the menu
/// bar). Commit any composition first.
public func setEnglishMode(_ on: Bool) {
guard on != isEnglishMode else { return }
reset()
isEnglishMode = on
previousMode = on ? .literal : nil
previousCommitTime = now()
}
/// Forces the whole buffer into the other mode (a correction that is learned).
private func flip() {
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
+11 -2
View File
@@ -50,8 +50,8 @@ The Windows version runs as a module inside PIME, which users install
separately. No PIME code is included in this software.
4. CKIP GPT2 Tiny Chinese — AI 選字語言模型 (language model, macOS)
-------------------------------------------------------------------
4. CKIP GPT2 Tiny Chinese — AI 選字語言模型 (language model)
-----------------------------------------------------------
https://huggingface.co/ckiplab/gpt2-tiny-chinese
https://github.com/ckiplab/ckip-transformers
Author: Mu Yang at CKIP, Academia Sinica (中央研究院 詞庫小組)
@@ -62,3 +62,12 @@ lm-vocab.txt is its vocabulary (from bert-base-chinese). They are used unmodifie
otherwise, to rerank conversion candidates ("AI 選字"). The source code of this input
method, including the conversion script and the code that runs the model, is
available from the project repository.
5. NumPy — 數值運算 (Windows / Linux, for AI 選字)
------------------------------------------------
https://numpy.org — BSD 3-Clause License (and the licenses of the libraries it bundles,
such as OpenBLAS). The Windows package includes unmodified NumPy wheels for 32-bit
Python 3.8 and 3.15 under autozhuyin/vendor/; the Linux installer downloads the
matching wheel from PyPI when the system has no NumPy. Each copy keeps its license
files in its numpy-*.dist-info folder.
@@ -133,6 +133,16 @@ final class EngineTests: XCTestCase {
return out
}
/// The menu-bar input mode can switch English mode directly.
func testSetEnglishMode() {
let e = makeEngine()
e.setEnglishMode(true)
XCTAssertTrue(e.isEnglishMode)
XCTAssertEqual(e.handle(.character("a")).handled, false) // goes to the app
e.setEnglishMode(false)
XCTAssertEqual(type("su3cl3 ", into: e), "你好")
}
func testMixedSentence() {
let e = makeEngine()
// 我 + "hello world " + 你好
+80
View File
@@ -17,6 +17,10 @@ import time
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
# numpy for "AI 選字", downloaded by install.sh when the system has none.
_VENDOR = os.path.join(HERE, "vendor", "cp%d%d" % sys.version_info[:2])
if os.path.isdir(_VENDOR):
sys.path.insert(1, _VENDOR)
import gi # noqa: E402
@@ -24,7 +28,9 @@ gi.require_version("IBus", "1.0")
from gi.repository import GLib, GObject, IBus # noqa: E402
import autozhuyin # noqa: E402
from autozhuyin.core.ai import AI, MODE_NAMES # noqa: E402
from autozhuyin.core.history import UserHistory # noqa: E402
from autozhuyin.core.reranker import MODES # noqa: E402
from autozhuyin.core.classifier import LITERAL, Classifier, LearnedPreferences # noqa: E402
from autozhuyin.core.engine import ( # noqa: E402
BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, SHIFT_TAP, TAB, UP, CompositionEngine,
@@ -50,6 +56,15 @@ except OSError:
pass
_classifier = None
_ai = None
def shared_ai():
""""AI 選字": settings and the language model, loaded in the background once."""
global _ai
if _ai is None:
_ai = AI(_DATA, _USER_DIR)
return _ai
def _save_json(path, data):
@@ -118,6 +133,7 @@ class AutoZhuyinEngine(IBus.Engine):
if getattr(self, "engine", None) is not None:
return
self.engine = CompositionEngine(shared_classifier())
self.engine.reranker = shared_ai().reranker
self.shift_tap = ShiftTapDetector()
self.table = IBus.LookupTable.new(self.engine.page_size, 0, True, False)
self.table.set_orientation(IBus.Orientation.VERTICAL)
@@ -127,8 +143,43 @@ class AutoZhuyinEngine(IBus.Engine):
symbol=IBus.Text.new_from_string("自"),
tooltip=IBus.Text.new_from_string("單按 Shift 切換英文模式"),
sensitive=True, visible=True, state=IBus.PropState.UNCHECKED)
# "AI 選字" menu: modes (radio), auto-pause (toggle), status, clear learned data.
ai = shared_ai()
self.ai_props = {}
sub = IBus.PropList()
for mode in MODES:
prop = IBus.Property(
key="ai." + mode, prop_type=IBus.PropType.RADIO,
label=IBus.Text.new_from_string("AI 選字:" + MODE_NAMES[mode]),
tooltip=IBus.Text.new_from_string(""), sensitive=True, visible=True,
state=IBus.PropState.CHECKED if ai.reranker.mode == mode else IBus.PropState.UNCHECKED)
self.ai_props[mode] = prop
sub.append(prop)
self.pause_prop = IBus.Property(
key="ai.autopause", prop_type=IBus.PropType.TOGGLE,
label=IBus.Text.new_from_string("系統忙碌時自動暫停 AI"),
tooltip=IBus.Text.new_from_string(""), sensitive=True, visible=True,
state=IBus.PropState.CHECKED if ai.auto_pause else IBus.PropState.UNCHECKED)
self.status_prop = IBus.Property(
key="ai.status", prop_type=IBus.PropType.NORMAL,
label=IBus.Text.new_from_string(ai.status_text()),
tooltip=IBus.Text.new_from_string(""), sensitive=False, visible=True,
state=IBus.PropState.UNCHECKED)
self.clear_prop = IBus.Property(
key="clear", prop_type=IBus.PropType.NORMAL,
label=IBus.Text.new_from_string("清除學習紀錄"),
tooltip=IBus.Text.new_from_string(""), sensitive=True, visible=True,
state=IBus.PropState.UNCHECKED)
for prop in (self.pause_prop, self.status_prop, self.clear_prop):
sub.append(prop)
self.ai_menu = IBus.Property(
key="ai", prop_type=IBus.PropType.MENU,
label=IBus.Text.new_from_string("設定"),
tooltip=IBus.Text.new_from_string("AI 選字與學習紀錄"),
sensitive=True, visible=True, state=IBus.PropState.UNCHECKED, sub_props=sub)
self.props = IBus.PropList()
self.props.append(self.mode_prop)
self.props.append(self.ai_menu)
# --- keys ---
@@ -211,12 +262,38 @@ class AutoZhuyinEngine(IBus.Engine):
self._update(None)
return True
# --- settings menu ---
def do_property_activate(self, prop_name, state):
self._setup()
ai = shared_ai()
if prop_name.startswith("ai.") and prop_name[3:] in MODES:
if state == IBus.PropState.CHECKED:
ai.set_mode(prop_name[3:])
elif prop_name == "ai.autopause":
ai.set_auto_pause(state == IBus.PropState.CHECKED)
elif prop_name == "clear":
shared_classifier().learned.clear()
shared_classifier().history.clear()
self._refresh_ai_props()
def _refresh_ai_props(self):
ai = shared_ai()
for mode, prop in self.ai_props.items():
prop.set_state(IBus.PropState.CHECKED if ai.reranker.mode == mode else IBus.PropState.UNCHECKED)
self.update_property(prop)
self.pause_prop.set_state(IBus.PropState.CHECKED if ai.auto_pause else IBus.PropState.UNCHECKED)
self.update_property(self.pause_prop)
self.status_prop.set_label(IBus.Text.new_from_string(ai.status_text()))
self.update_property(self.status_prop)
# --- lifecycle ---
def do_focus_in(self):
self._setup()
self.register_properties(self.props)
self._update_mode()
self._refresh_ai_props()
def do_focus_out(self):
self._setup()
@@ -268,6 +345,9 @@ class AutoZhuyinEngine(IBus.Engine):
english = self.engine.english_mode
self.mode_prop.set_label(IBus.Text.new_from_string("英文" if english else "中英自動"))
self.mode_prop.set_symbol(IBus.Text.new_from_string("英" if english else "自"))
icon = os.path.join(HERE, "icon-en.png" if english else "icon.png")
if os.path.exists(icon):
self.mode_prop.set_icon(icon) # panels that show property icons
self.update_property(self.mode_prop)
+11 -1
View File
@@ -33,13 +33,23 @@
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字;
選過的字會被記住,下次優先出現
Esc 取消組字
單按 Shift 切換英文模式(按鍵直接送出),再單按一次回到中英自動判斷
單按 Shift 切換英文模式(按鍵直接送出,狀態圖示變成「英」),再單按一次回到中英自動判斷
Shift+字母 打大寫字母(也會強制英文)
` + 標點 強制全形標點,例如 `< 或 `, = , `. = 。 `/ = ? `; = ;
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
AI 選字
-------
用中研院 CKIP 的繁體中文小型語言模型,看前後文挑同音字(他跑「得」很快、請你「再」說一次),
完全在本機運算。需要 numpy:系統沒有的話,install.sh 會從 PyPI 下載符合的版本(約 15 MB)。
在 IBus/系統匣的輸入法選單「設定」裡可以選:
AI 選字:關閉/省電/標準(預設)/高品質
系統忙碌時自動暫停 AI(預設開啟:CPU 很忙或過熱時暫停,省電設定或電量低時改用省電)
設定存在 ~/.config/autozhuyin/settings.json。
切換與移除
----------
暫時改回一般鍵盤: ibus engine xkb:us::eng
+21 -3
View File
@@ -23,13 +23,31 @@ fi
# 2. Copy the program.
pkill -f "autozhuyin/autozhuyin_ibus.py" 2>/dev/null || true
VENDOR_BACKUP=""
if [ -d "$DEST/vendor" ]; then # keep numpy downloaded by an earlier install
VENDOR_BACKUP="$(mktemp -d)"
mv "$DEST/vendor" "$VENDOR_BACKUP/"
fi
rm -rf "$DEST"
mkdir -p "$DEST"
cp -R autozhuyin autozhuyin_ibus.py version.txt "$DEST/"
[ -f icon.png ] && cp icon.png "$DEST/"
[ -f THIRD_PARTY_LICENSES.txt ] && cp THIRD_PARTY_LICENSES.txt "$DEST/"
cp -R autozhuyin autozhuyin_ibus.py version.txt tools "$DEST/"
for f in icon.png icon-en.png THIRD_PARTY_LICENSES.txt GPL-3.0.txt; do [ -f "$f" ] && cp "$f" "$DEST/"; done
[ -n "$VENDOR_BACKUP" ] && mv "$VENDOR_BACKUP/vendor" "$DEST/" && rmdir "$VENDOR_BACKUP"
chmod +x "$DEST/autozhuyin_ibus.py"
# 2b. numpy for "AI 選字": use the system's, or download the matching version from PyPI.
TAG="$("$PY" -c 'import sys; print("cp%d%d" % sys.version_info[:2])')"
if "$PY" -c 'import numpy' 2>/dev/null; then
echo "AI 選字:使用系統的 numpy。"
elif [ -d "$DEST/vendor/$TAG/numpy" ]; then
echo "AI 選字:使用先前下載的 numpy。"
else
echo "AI 選字需要 numpy,正在從 PyPI 下載(約 15 MB)…"
if ! "$PY" -I "$DEST/tools/fetch_numpy.py" "$DEST/vendor" auto auto; then
echo "下載 numpy 失敗:輸入法照常可用,只是沒有 AI 選字。之後可以重新執行 install.sh。"
fi
fi
# 3. Start with the desktop session: registers the engine with IBus and selects it.
mkdir -p "$AUTOSTART"
cat > "$AUTOSTART/autozhuyin.desktop" <<DESKTOP
+21 -4
View File
@@ -54,9 +54,11 @@ class LookupTable:
class Property:
def __init__(self, **kw): self.label = kw["label"].text
def __init__(self, **kw): self.key, self.label, self.state = kw["key"], kw["label"].text, kw.get("state", 0)
def set_label(self, t): self.label = t.text
def set_symbol(self, t): pass
def set_state(self, s): self.state = s
def set_icon(self, path): self.icon = path
class Engine:
@@ -71,7 +73,9 @@ class Engine:
def update_lookup_table(self, table, visible): self.shown = list(table.items)
def hide_lookup_table(self): self.shown = None
def register_properties(self, p): pass
def update_property(self, p): self.mode_label = p.label
def update_property(self, p):
if p.key == "mode":
self.mode_label = p.label
IBus.Text, IBus.AttrList, IBus.LookupTable, IBus.Property, IBus.Engine = Text, AttrList, LookupTable, Property, Engine
@@ -80,8 +84,8 @@ IBus.attr_underline_new = lambda kind, a, b: ("underline", a, b)
IBus.attr_foreground_new = lambda color, a, b: ("color", a, b)
IBus.AttrUnderline = types.SimpleNamespace(SINGLE=1)
IBus.Orientation = types.SimpleNamespace(VERTICAL=1)
IBus.PropType = types.SimpleNamespace(NORMAL=0)
IBus.PropState = types.SimpleNamespace(UNCHECKED=0)
IBus.PropType = types.SimpleNamespace(NORMAL=0, TOGGLE=1, RADIO=2, MENU=3)
IBus.PropState = types.SimpleNamespace(UNCHECKED=0, CHECKED=1)
IBus.PreeditFocusMode = types.SimpleNamespace(COMMIT=1)
gi = types.ModuleType("gi")
@@ -173,6 +177,19 @@ class IBusEngineTests(unittest.TestCase):
press(self.e, "su3 \n")
self.assertTrue(self.e.committed.endswith(second))
def test_ai_menu(self):
e = self.e
self.assertEqual(e.ai_props["standard"].state, 1)
e.do_property_activate("ai.saver", 1)
self.assertEqual(autozhuyin_ibus.shared_ai().reranker.mode, "saver")
self.assertEqual(e.ai_props["saver"].state, 1)
self.assertEqual(e.ai_props["standard"].state, 0)
e.do_property_activate("ai.autopause", 0)
self.assertFalse(autozhuyin_ibus.shared_ai().auto_pause)
self.assertTrue(e.status_prop.label)
e.do_property_activate("ai.standard", 1)
e.do_property_activate("ai.autopause", 1)
def test_focus_out_resets(self):
press(self.e, "su3")
self.e.do_focus_out()
+5
View File
@@ -18,6 +18,8 @@ rm -rf "$APP"
mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
cp "$BIN" "$APP/Contents/MacOS/AutoZhuyin"
cp Resources/Info.plist "$APP/Contents/Info.plist"
# Localized names of the input method and its modes (System Settings, input menu).
cp -R Resources/*.lproj "$APP/Contents/Resources/"
cp Resources/lexicon.txt Resources/english-common.txt Resources/classifier-weights.json "$APP/Contents/Resources/"
cp THIRD_PARTY_LICENSES.txt "$APP/Contents/Resources/"
# "AI 選字" language model (CKIP gpt2-tiny-chinese, GPL-3.0), fetched and converted once.
@@ -31,6 +33,9 @@ if [ ! -f build/icon.tiff ]; then
xcrun swift scripts/make-icon.swift build/icon.tiff
fi
cp build/icon.tiff "$APP/Contents/Resources/icon.tiff"
# English-mode icon ("A"), shown in the menu bar while English mode is on.
[ -f build/icon-en.tiff ] || xcrun swift scripts/make-icon.swift build/icon-en.tiff A
cp build/icon-en.tiff "$APP/Contents/Resources/icon-en.tiff"
codesign --force --sign - "$APP"
echo "Built $APP"
+4 -3
View File
@@ -1,8 +1,9 @@
// Renders the menu-bar icon ("注" with a small "A") to a TIFF.
// Usage: swift scripts/make-icon.swift <output.tiff>
// Renders a menu-bar icon (one character, "注" by default; "A" for English mode) to a TIFF.
// Usage: swift scripts/make-icon.swift <output.tiff> [character]
import AppKit
let out = CommandLine.arguments.count > 1 ? CommandLine.arguments[1] : "icon.tiff"
let glyph = CommandLine.arguments.count > 2 ? CommandLine.arguments[2] : "注"
let reps: [NSBitmapImageRep] = [16, 32].map { size in
let rep = NSBitmapImageRep(bitmapDataPlanes: nil, pixelsWide: size, pixelsHigh: size,
bitsPerSample: 8, samplesPerPixel: 4, hasAlpha: true, isPlanar: false,
@@ -11,7 +12,7 @@ let reps: [NSBitmapImageRep] = [16, 32].map { size in
NSGraphicsContext.saveGraphicsState()
NSGraphicsContext.current = NSGraphicsContext(bitmapImageRep: rep)
let font = NSFont.systemFont(ofSize: 13, weight: .semibold)
let text = NSAttributedString(string: "注", attributes: [.font: font, .foregroundColor: NSColor.black])
let text = NSAttributedString(string: glyph, attributes: [.font: font, .foregroundColor: NSColor.black])
let s = text.size()
text.draw(at: NSPoint(x: (16 - s.width) / 2, y: (16 - s.height) / 2))
NSGraphicsContext.restoreGraphicsState()
+6 -1
View File
@@ -8,6 +8,7 @@ VERSION="$(python3 -I -c 'import json,sys; print(json.load(open(sys.argv[1]))["v
NAME="AutoZhuyin-Linux-$VERSION"
OUT="build/$NAME"
[ -s build/lm/lm.bin ] || scripts/fetch-lm.sh >/dev/null # "AI 選字" model
python3 -I windows/tools/build_data.py
(cd windows && python3 -m unittest discover -q tests)
python3 -m unittest discover -q -s linux/tests
@@ -21,9 +22,13 @@ mkdir -p "$OUT/autozhuyin"
rsync -a --exclude __pycache__ --exclude '*.pyc' \
windows/autozhuyin/__init__.py windows/autozhuyin/core windows/autozhuyin/data "$OUT/autozhuyin/"
cp linux/autozhuyin_ibus.py linux/installer/install.sh linux/installer/uninstall.sh linux/installer/README.txt \
THIRD_PARTY_LICENSES.txt "$OUT/"
THIRD_PARTY_LICENSES.txt licenses/GPL-3.0.txt "$OUT/"
mkdir -p "$OUT/tools"
cp windows/tools/fetch_numpy.py "$OUT/tools/" # install.sh downloads numpy if the system has none
echo "$VERSION" > "$OUT/version.txt"
sips -s format png -z 64 64 build/icon.tiff --out "$OUT/icon.png" >/dev/null
[ -f build/icon-en.tiff ] || DEVELOPER_DIR="${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}" xcrun swift scripts/make-icon.swift build/icon-en.tiff A
sips -s format png -z 64 64 build/icon-en.tiff --out "$OUT/icon-en.png" >/dev/null
chmod +x "$OUT/install.sh" "$OUT/uninstall.sh" "$OUT/autozhuyin_ibus.py"
# Plain ustar without macOS extended attributes, so GNU tar doesn't warn.
+13
View File
@@ -8,6 +8,7 @@ VERSION="$(python3 -I -c 'import json,sys; print(json.load(open(sys.argv[1]))["v
NAME="AutoZhuyin-Windows-$VERSION"
OUT="build/$NAME"
[ -s build/lm/lm.bin ] || scripts/fetch-lm.sh >/dev/null # "AI 選字" model
python3 -I windows/tools/build_data.py
(cd windows && python3 -m unittest discover -q tests)
@@ -17,6 +18,10 @@ if [ ! -f build/icon.tiff ]; then
fi
sips -s format png -z 32 32 build/icon.tiff --out build/icon-32.png >/dev/null
python3 -I windows/tools/png_to_ico.py build/icon-32.png windows/autozhuyin/icon.ico
# English-mode icon ("A") for the taskbar while English mode is on.
[ -f build/icon-en.tiff ] || DEVELOPER_DIR="${DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}" xcrun swift scripts/make-icon.swift build/icon-en.tiff A
sips -s format png -z 32 32 build/icon-en.tiff --out build/icon-en-32.png >/dev/null
python3 -I windows/tools/png_to_ico.py build/icon-en-32.png windows/autozhuyin/icon-en.ico
rm -rf "$OUT" "$OUT.zip"
mkdir -p "$OUT"
@@ -29,6 +34,14 @@ done
# Third-party notices: next to the installer and inside the installed folder.
sed 's/$/\r/' THIRD_PARTY_LICENSES.txt > "$OUT/THIRD_PARTY_LICENSES.txt"
cp "$OUT/THIRD_PARTY_LICENSES.txt" "$OUT/autozhuyin/"
sed 's/$/\r/' licenses/GPL-3.0.txt > "$OUT/autozhuyin/GPL-3.0.txt"
# numpy for "AI 選字", for both Pythons PIME ships (1.3: 3.8, 2.x: 3.15; both 32-bit).
for tag in cp38 cp315; do
[ -d "build/numpy-vendor/$tag" ] || python3 -I windows/tools/fetch_numpy.py build/numpy-vendor "$tag" win32
done
mkdir -p "$OUT/autozhuyin/vendor"
cp -R build/numpy-vendor/cp38 build/numpy-vendor/cp315 "$OUT/autozhuyin/vendor/"
(cd build && zip -qr -X "$NAME.zip" "$NAME")
echo "Built build/$NAME.zip"
+84 -2
View File
@@ -6,12 +6,21 @@ tests); this file only translates PIME key events and replies.
"""
import json
import os
import sys
import time
from keycodes import * # noqa: F401,F403 (VK_* constants, provided by PIME)
from textService import TextService
# numpy for "AI 選字" is bundled per Python version (PIME 1.3 has 3.8, PIME 2 has 3.15).
_VENDOR = os.path.join(os.path.dirname(os.path.abspath(__file__)), "vendor",
"cp%d%d" % sys.version_info[:2])
if os.path.isdir(_VENDOR) and _VENDOR not in sys.path:
sys.path.insert(0, _VENDOR)
from keycodes import * # noqa: F401,F403 (VK_* constants, provided by PIME)
from textService import COMMAND_LEFT_CLICK, COMMAND_MENU, TextService # noqa: F401
from .core.ai import AI, MODE_NAMES
from .core.history import UserHistory
from .core.reranker import MODES
from .core.classifier import Classifier, LearnedPreferences
from .core.engine import BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, SHIFT_TAP, TAB, UP, CompositionEngine
from .core.english import EnglishModel
@@ -26,6 +35,12 @@ _USER_PHRASES = os.path.join(_USER_DIR, "user-phrases.txt")
_HISTORY_FILE = os.path.join(_USER_DIR, "user-history.json")
_classifier = None
_ai = None
ID_MODE_ICON, ID_AUTO_PAUSE, ID_STATUS, ID_CLEAR = 1, 2, 3, 4
ID_AI_MODE = {mode: 10 + i for i, mode in enumerate(MODES)} # 10..13
_ICON = os.path.join(_HERE, "icon.ico")
_ICON_EN = os.path.join(_HERE, "icon-en.ico") # shown while English mode is on
def _save_json(path, data):
@@ -72,6 +87,14 @@ def _shared_classifier():
return _classifier
def _shared_ai():
""""AI 選字": settings and the language model, loaded in the background once."""
global _ai
if _ai is None:
_ai = AI(_DATA, _USER_DIR)
return _ai
_SPECIAL = {
VK_BACK: BACKSPACE, VK_RETURN: ENTER, VK_ESCAPE: ESCAPE, VK_TAB: TAB, # noqa: F405
VK_UP: UP, VK_DOWN: DOWN, VK_LEFT: LEFT, VK_RIGHT: RIGHT, # noqa: F405
@@ -83,6 +106,7 @@ class AutoZhuyinTextService(TextService):
TextService.__init__(self, client)
self.engine = CompositionEngine(_shared_classifier())
self.shift_tap = ShiftTapDetector()
self.engine.reranker = _shared_ai().reranker
# --- lifecycle ---
@@ -90,11 +114,68 @@ class AutoZhuyinTextService(TextService):
TextService.onActivate(self)
self.setSelKeys("123456789")
self.customizeUI(candFontSize=16, candPerRow=1, candUseCursor=True)
# Taskbar IME icon (Windows 8+) and language bar button; both open the settings menu.
self._icon_english = self.engine.english_mode
self.addButton("windows-mode-icon", icon=self._mode_icon(), tooltip=self._mode_tooltip(),
commandId=ID_MODE_ICON)
self.addButton("autozhuyin-settings", icon=_ICON, tooltip="自動注音設定", type="menu")
def onDeactivate(self):
self.engine.reset()
self.removeButton("windows-mode-icon")
self.removeButton("autozhuyin-settings")
TextService.onDeactivate(self)
# --- taskbar icon: "注" for 中英自動, "A" for English mode ---
def _mode_icon(self):
return _ICON_EN if self.engine.english_mode and os.path.exists(_ICON_EN) else _ICON
def _mode_tooltip(self):
return ("英文模式" if self.engine.english_mode else "中英自動") + "(左鍵切換,右鍵設定)"
def _update_mode_icon(self):
if getattr(self, "_icon_english", None) == self.engine.english_mode:
return
self._icon_english = self.engine.english_mode
self.changeButton("windows-mode-icon", icon=self._mode_icon(), tooltip=self._mode_tooltip())
# --- menu (AI 選字 settings) ---
def onMenu(self, buttonId):
if buttonId not in ("windows-mode-icon", "autozhuyin-settings"):
return None
ai = _shared_ai()
items = [{"text": "AI 選字:" + MODE_NAMES[m], "id": ID_AI_MODE[m], "checked": ai.reranker.mode == m}
for m in MODES]
items += [
{},
{"text": "系統忙碌時自動暫停 AI", "id": ID_AUTO_PAUSE, "checked": ai.auto_pause},
{"text": ai.status_text(), "id": ID_STATUS, "enabled": False},
{},
{"text": "清除學習紀錄", "id": ID_CLEAR},
]
return items
def onCommand(self, commandId, commandType):
ai = _shared_ai()
if commandId == ID_MODE_ICON and commandType == COMMAND_LEFT_CLICK and not self.engine.keys:
self.engine.handle(SHIFT_TAP) # left click: English mode on / off
self._update_mode_icon()
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
return
for mode, cid in ID_AI_MODE.items():
if commandId == cid:
ai.set_mode(mode)
self.showMessage("AI 選字:" + MODE_NAMES[mode], 1)
return
if commandId == ID_AUTO_PAUSE:
ai.set_auto_pause(not ai.auto_pause)
elif commandId == ID_CLEAR:
_shared_classifier().learned.clear()
_shared_classifier().history.clear()
self.showMessage("已清除學習紀錄", 1)
def onCompositionTerminated(self, forced):
# The app ended the composition (focus change, mouse click): the shown text stays.
self.engine.reset()
@@ -155,6 +236,7 @@ class AutoZhuyinTextService(TextService):
return False
self.engine.handle(SHIFT_TAP)
self._update(None)
self._update_mode_icon()
if not self.engine.keys:
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
return True
+108
View File
@@ -0,0 +1,108 @@
""""AI 選字" for the Windows and Linux shells: settings, background model loading,
system-load detection and the status line shown in the menu."""
import json
import os
import threading
from . import lm
from .reranker import (MODES, NORMAL, OFF, ST_ACTIVE, ST_BUSY, ST_LOADING, ST_OFF, ST_SLOW,
STANDARD, Reranker)
from .system_monitor import REDUCED, SystemMonitor
MODE_NAMES = {OFF: "關閉", "saver": "省電", STANDARD: "標準", "quality": "高品質"}
class AI:
def __init__(self, data_dir, user_dir, monitor=None, background=True):
self.model_path = os.path.join(data_dir, "lm.bin")
self.vocab_path = os.path.join(data_dir, "lm-vocab.txt")
self.settings_path = os.path.join(user_dir, "settings.json")
self.monitor = monitor or SystemMonitor()
self.background = background
settings = self._load_settings()
mode = settings.get("ai_mode", STANDARD)
self.auto_pause = bool(settings.get("ai_auto_pause", True))
self.reranker = Reranker(mode if mode in MODES else STANDARD)
self.reranker.system_state = lambda: self.monitor.state() if self.auto_pause else NORMAL
self.error = None # why the model cannot be used, if it cannot
self._loading = False
self.load_model()
# --- settings ---
def _load_settings(self):
try:
with open(self.settings_path, encoding="utf-8") as f:
data = json.load(f)
return data if isinstance(data, dict) else {}
except (OSError, ValueError):
return {}
def _save_settings(self):
data = self._load_settings()
data.update(ai_mode=self.reranker.mode, ai_auto_pause=self.auto_pause)
try:
os.makedirs(os.path.dirname(self.settings_path), exist_ok=True)
with open(self.settings_path, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False, indent=1)
except OSError:
pass
def set_mode(self, mode):
if mode not in MODES:
return
self.reranker.mode = mode
if mode == OFF:
self.reranker.model = None # free the memory
else:
self.load_model()
self._save_settings()
def set_auto_pause(self, on):
self.auto_pause = bool(on)
self._save_settings()
# --- model ---
def load_model(self):
if self.reranker.mode == OFF or self.reranker.model is not None or self._loading or self.error:
return
if not lm.available():
self.error = "需要 numpy"
return
if not (os.path.exists(self.model_path) and os.path.exists(self.vocab_path)):
self.error = "AI 模型未安裝"
return
self._loading = True
def work():
try:
model = lm.GPT2LanguageModel(self.model_path, self.vocab_path)
except Exception as e: # noqa: BLE001 — any failure just disables AI
self.error = f"AI 模型載入失敗:{e}"
model = None
self._loading = False
if self.reranker.mode != OFF:
self.reranker.model = model
if self.background:
threading.Thread(target=work, daemon=True).start()
else:
work()
def status_text(self):
if self.error:
return self.error
self.reranker.effective_mode() # refresh the status
st = self.reranker.status
if st == ST_OFF:
return "AI 選字已關閉"
if st == ST_LOADING:
return "AI 模型載入中…"
if st == ST_BUSY:
return "AI 已暫停:系統忙碌或過熱"
if st == ST_SLOW:
return "AI 已暫停:運算太慢,一分鐘後再試"
if st == ST_ACTIVE and self.auto_pause and self.monitor.state() == REDUCED:
return "AI 使用中(省電模式)"
return "AI 使用中"
+10
View File
@@ -65,6 +65,8 @@ class CompositionEngine:
self.inserting = []
# Index of a held space (see HOLD_MIN_SCORE), or None.
self.held = None
self.reranker = None # "AI 選字" (reranker.Reranker); None = lexicon only
self._recent_text = "" # last characters committed: context for the reranker
self._previous_mode = None
self._previous_time = float("-inf")
self._cache_key = None
@@ -159,8 +161,15 @@ class CompositionEngine:
def _conversion(self, seg, final):
syl, pending = self._convertible(seg, final)
fx = {k: v for k, v in self.fixed.get(seg[0], {}).items() if k[1] <= len(syl)}
if self.reranker is not None:
chosen = self.reranker.choose(self.converter, syl, fx, self._rerank_context())
if chosen is not None:
return chosen, pending
return self.converter.convert(syl, fx), pending
def _rerank_context(self):
return self._recent_text if self.now() - self._previous_time < self.context_window else ""
def _render(self, seg, final):
start, end, mode = seg
if mode == LITERAL:
@@ -556,6 +565,7 @@ class CompositionEngine:
segs = segs if segs is not None else self.segments
text = "".join(self._render(s, True) for s in segs)
self._remember_picks(segs)
self._recent_text = (self._rerank_context() + text)[-20:]
if self.forced_mode:
self.classifier.learned.record("".join(self.keys), self.forced_mode)
self._previous_mode = segs[-1][2] if segs else None
+22
View File
@@ -130,6 +130,28 @@ class Converter:
j = seg[0][0]
return segs[::-1]
def kbest(self, syllables, k, fixed=None):
"""The k best paths [(log10 score, [((start, end), text)])], best first."""
fixed = fixed or {}
n = len(syllables)
if n == 0 or k <= 0:
return []
max_len = max(1, self.lexicon.max_phrase_length)
beams = [[] for _ in range(n + 1)]
beams[0] = [(0.0, [])]
for j in range(1, n + 1):
cand = []
for length in range(1, min(max_len, j) + 1):
i = j - length
if not beams[i]:
continue
for text, score in self._options(syllables, i, j, fixed):
for s, segs in beams[i]:
cand.append((s + score, segs + [((i, j), text)]))
cand.sort(key=lambda e: -e[0])
beams[j] = cand[:k]
return beams[n]
def candidates(self, syllables, end):
"""[((start, end), text)] for phrases ending at syllable `end`, longest first."""
if end <= 0 or end > len(syllables):
+109
View File
@@ -0,0 +1,109 @@
"""GPT-2 language model for "AI 選字" (numpy). Port of LanguageModel.swift.
Reads the "AZLM" file made by training/lm/export_lm.py (CKIP gpt2-tiny-chinese, GPL-3.0,
float16 weights, expanded to float32 in memory). numpy is optional: when it cannot be
imported, `available()` is False and the input method works without AI.
"""
import json
import math
import struct
try:
import numpy as np
except Exception: # ImportError, or a numpy build for another Python / architecture
np = None
CLS, UNK = 101, 100
def available():
return np is not None
class GPT2LanguageModel:
max_tokens = 64
def __init__(self, model_path, vocab_path):
if np is None:
raise RuntimeError("numpy is not available")
with open(model_path, "rb") as f:
raw = f.read()
if raw[:4] != b"AZLM":
raise ValueError("not an AZLM file")
n = struct.unpack("<I", raw[4:8])[0]
header = json.loads(raw[8:8 + n])
data = np.frombuffer(raw, dtype="<f2", offset=8 + n).astype(np.float32)
self.n_layer, self.n_head = header["n_layer"], header["n_head"]
self.d, self.eps = header["n_embd"], header["eps"]
self.w = {}
for name, shape, offset in header["tensors"]:
size = int(np.prod(shape))
self.w[name] = data[offset:offset + size].reshape(shape)
self.vocab = {}
with open(vocab_path, encoding="utf-8") as f:
for i, line in enumerate(f):
token = line.rstrip("\n")
if len(token) == 1:
self.vocab.setdefault(token, i)
def ids(self, text):
return [self.vocab.get(c, UNK) for c in text]
def _ln(self, x, name):
mu = x.mean(-1, keepdims=True)
var = ((x - mu) ** 2).mean(-1, keepdims=True)
return (x - mu) / np.sqrt(var + self.eps) * self.w[name + ".weight"] + self.w[name + ".bias"]
def _hidden(self, ids):
"""ids [B, T] → final hidden states [B, T, d]."""
w = self.w
B, T = ids.shape
x = w["wte.weight"][ids] + w["wpe.weight"][:T]
mask = np.triu(np.full((T, T), -1e9, dtype=np.float32), 1)
hd = self.d // self.n_head
for l in range(self.n_layer):
p = f"h.{l}."
h = self._ln(x, p + "ln_1")
qkv = h @ w[p + "attn.c_attn.weight"] + w[p + "attn.c_attn.bias"]
q, k, v = np.split(qkv, 3, axis=-1)
q = q.reshape(B, T, self.n_head, hd).transpose(0, 2, 1, 3)
k = k.reshape(B, T, self.n_head, hd).transpose(0, 2, 3, 1)
v = v.reshape(B, T, self.n_head, hd).transpose(0, 2, 1, 3)
att = q @ k / math.sqrt(hd) + mask
att = np.exp(att - att.max(-1, keepdims=True))
att /= att.sum(-1, keepdims=True)
a = (att @ v).transpose(0, 2, 1, 3).reshape(B, T, self.d)
x = x + a @ w[p + "attn.c_proj.weight"] + w[p + "attn.c_proj.bias"]
h = self._ln(x, p + "ln_2")
h = h @ w[p + "mlp.c_fc.weight"] + w[p + "mlp.c_fc.bias"]
h = 0.5 * h * (1 + np.tanh(math.sqrt(2 / math.pi) * (h + 0.044715 * h ** 3))) # gelu_new
x = x + h @ w[p + "mlp.c_proj.weight"] + w[p + "mlp.c_proj.bias"]
return self._ln(x, "ln_f")
def log_probs(self, texts, context=""):
"""Natural-log probabilities of several texts after the same context. Texts of the
same length are scored together in one batch."""
out = [0.0] * len(texts)
by_len = {}
for i, t in enumerate(texts):
if t:
by_len.setdefault(len(t), []).append(i)
for length, idx in by_len.items():
ctx = self.ids(context)
if len(ctx) + length + 1 > self.max_tokens:
ctx = ctx[max(0, len(ctx) - (self.max_tokens - length - 1)):]
targets = np.array([self.ids(texts[i]) for i in idx]) # [B, L]
prefix = np.array([CLS] + ctx)
inputs = np.concatenate([np.tile(prefix, (len(idx), 1)), targets[:, :-1]], axis=1)
h = self._hidden(inputs)[:, len(ctx):, :] # rows predicting targets
logits = h @ self.w["wte.weight"].T # [B, L, V]
m = logits.max(-1, keepdims=True)
lse = np.log(np.exp(logits - m).sum(-1)) + m[..., 0]
picked = np.take_along_axis(logits, targets[..., None], -1)[..., 0]
scores = (picked - lse).sum(-1)
for j, i in enumerate(idx):
out[i] = float(scores[j])
return out
def log_prob(self, text, context=""):
return self.log_probs([text], context)[0]
+102
View File
@@ -0,0 +1,102 @@
""""AI 選字": reranks the converter's best paths with a language model. Port of Reranker.swift.
score = lexicon score (log10 → ln) + λ · LM log-probability
Typing is never blocked for long: a rerank that runs over its time budget counts as an
overrun, and three in a row pause AI for a minute. `system_state` lets the shell pause
or reduce it when the machine is busy, hot or saving power.
"""
import math
import time
OFF, SAVER, STANDARD, QUALITY = "off", "saver", "standard", "quality"
MODES = (OFF, SAVER, STANDARD, QUALITY)
# Paths reranked, and the time budget (seconds) of one rerank. numpy is slower than
# the Mac's Accelerate code, so the budgets are larger than in Reranker.swift.
K = {SAVER: 3, STANDARD: 10, QUALITY: 30}
BUDGET = {SAVER: 0.04, STANDARD: 0.08, QUALITY: 0.2}
NORMAL, REDUCED, PAUSED = "normal", "reduced", "paused" # system states
ST_OFF, ST_LOADING, ST_ACTIVE, ST_BUSY, ST_SLOW = "off", "loading", "active", "paused-busy", "paused-slow"
PAUSE_AFTER_OVERRUNS = 3
SLOW_PAUSE = 60.0
CACHE_LIMIT = 512
class Reranker:
def __init__(self, mode=STANDARD, model=None, now=time.monotonic):
self._mode = mode
self._model = model
self.lam = 0.6
self.system_state = lambda: NORMAL
self.now = now
self.status = ST_OFF
self._overruns = 0
self._paused_until = float("-inf")
self._cache = {}
@property
def mode(self):
return self._mode
@mode.setter
def mode(self, value):
self._mode = value
self._cache.clear()
@property
def model(self):
return self._model
@model.setter
def model(self, value):
self._model = value
self._cache.clear()
def effective_mode(self):
"""The mode used now (after the system state), or None when AI is not used."""
if self._mode == OFF:
self.status = ST_OFF
return None
if self._model is None:
self.status = ST_LOADING
return None
if self.now() < self._paused_until:
self.status = ST_SLOW
return None
state = self.system_state()
if state == PAUSED:
self.status = ST_BUSY
return None
self.status = ST_ACTIVE
return SAVER if state == REDUCED else self._mode
def choose(self, converter, syllables, fixed, context):
"""The reranked conversion [((start, end), text)], or None for the plain result."""
mode = self.effective_mode()
if mode is None or not syllables:
return None
key = (mode, context, tuple(s.reading for s in syllables), tuple(sorted(fixed.items())))
if key in self._cache:
return self._cache[key]
paths = converter.kbest(syllables, K[mode], fixed)
if len(paths) <= 1:
return paths[0][1] if paths else None
start = self.now()
lm = self._model.log_probs(["".join(t for _, t in segs) for _, segs in paths], context)
best = max(range(len(paths)), key=lambda i: paths[i][0] * math.log(10) + self.lam * lm[i])
if self.now() - start > BUDGET[mode]:
# Too slow: use this result (it is computed) but don't cache it, and pause AI
# after a few overruns in a row.
self._overruns += 1
if self._overruns >= PAUSE_AFTER_OVERRUNS:
self._paused_until = self.now() + SLOW_PAUSE
self._overruns = 0
self.status = ST_SLOW
return paths[best][1]
self._overruns = 0
if len(self._cache) >= CACHE_LIMIT:
self._cache.clear()
self._cache[key] = paths[best][1]
return paths[best][1]
+95
View File
@@ -0,0 +1,95 @@
"""Decides when "AI 選字" should step back (Windows and Linux; the Mac app has its own).
paused — CPU busy (over 90%), or the machine is very hot (Linux thermal zones ≥ 95 °C)
reduced — power saving (Windows battery saver; Linux low-power profile or battery ≤ 20%)
normal — otherwise. Checked at most every 2 seconds.
"""
import glob
import os
import sys
import time
from .reranker import NORMAL, PAUSED, REDUCED
class SystemMonitor:
interval = 2.0
busy_load = 0.9
def __init__(self, now=time.monotonic):
self.now = now
self._cached = (float("-inf"), NORMAL)
self._cpu_prev = None
def state(self):
t, s = self._cached
if self.now() - t < self.interval:
return s
try:
s = self._compute()
except Exception:
s = NORMAL
self._cached = (self.now(), s)
return s
def _compute(self):
if sys.platform == "win32":
return self._windows()
return self._linux()
# --- Linux ---
def _linux(self):
if os.getloadavg()[0] / max(1, os.cpu_count() or 1) > self.busy_load:
return PAUSED
temps = []
for path in glob.glob("/sys/class/thermal/thermal_zone*/temp"):
try:
temps.append(int(open(path).read().strip()) / 1000)
except (OSError, ValueError):
pass
if temps and max(temps) >= 95:
return PAUSED
try:
if open("/sys/firmware/acpi/platform_profile").read().strip() in ("low-power", "quiet"):
return REDUCED
except OSError:
pass
for bat in glob.glob("/sys/class/power_supply/BAT*"):
try:
status = open(bat + "/status").read().strip()
capacity = int(open(bat + "/capacity").read().strip())
except (OSError, ValueError):
continue
if status == "Discharging" and capacity <= 20:
return REDUCED
return NORMAL
# --- Windows ---
def _windows(self):
import ctypes
from ctypes import wintypes
class FILETIME(ctypes.Structure):
_fields_ = [("lo", wintypes.DWORD), ("hi", wintypes.DWORD)]
idle, kernel, user = FILETIME(), FILETIME(), FILETIME()
ctypes.windll.kernel32.GetSystemTimes(ctypes.byref(idle), ctypes.byref(kernel), ctypes.byref(user))
values = [(f.hi << 32) | f.lo for f in (idle, kernel, user)]
prev, self._cpu_prev = self._cpu_prev, values
if prev:
d_idle, d_kernel, d_user = (a - b for a, b in zip(values, prev))
total = d_kernel + d_user # kernel time includes idle time
if total > 0 and 1 - d_idle / total > self.busy_load:
return PAUSED
class SYSTEM_POWER_STATUS(ctypes.Structure):
_fields_ = [("ACLineStatus", ctypes.c_ubyte), ("BatteryFlag", ctypes.c_ubyte),
("BatteryLifePercent", ctypes.c_ubyte), ("SystemStatusFlag", ctypes.c_ubyte),
("BatteryLifeTime", wintypes.DWORD), ("BatteryFullLifeTime", wintypes.DWORD)]
status = SYSTEM_POWER_STATUS()
if ctypes.windll.kernel32.GetSystemPowerStatus(ctypes.byref(status)) and status.SystemStatusFlag == 1:
return REDUCED # battery saver on
return NORMAL
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "自動注音 AutoZhuyin",
"version": "0.11.0",
"version": "0.13.2",
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
"locale": "zh-Hant-TW",
"fallbackLocale": "zh-TW",
+10 -1
View File
@@ -31,7 +31,7 @@
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字;
選過的字會被記住,下次優先出現
Esc 取消組字
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線),
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線,工作列圖示變成 A),
再單按一次回到中英自動判斷,會出現「英文」/「中英自動」小提示;
打到一半時按,同 Tab 整段翻轉
Shift+字母 打大寫字母(也會強制英文)
@@ -43,6 +43,15 @@
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
AI 選字
-------
用中研院 CKIP 的繁體中文小型語言模型,看前後文挑同音字(他跑「得」很快、請你「再」說一次),
完全在本機運算,不連網。在工作列的輸入法圖示上按右鍵(或語言列的設定按鈕)可以選:
AI 選字:關閉/省電/標準(預設)/高品質
系統忙碌時自動暫停 AI(預設開啟:CPU 很忙時暫停,省電模式時改用省電)
選單裡會顯示目前狀態。電腦比較慢時建議用「省電」,運算太慢時也會自動暫停一分鐘。
回報問題
--------
請告訴我:
+100
View File
@@ -423,5 +423,105 @@ class UserHistoryTests(unittest.TestCase):
self.assertEqual(set(h.data["ㄊㄧㄢ"]), {"添", "田"})
class FakeLM:
"""Likes one string; `on_call` can advance a fake clock."""
def __init__(self, liked):
self.liked, self.calls, self.on_call = liked, 0, lambda: None
def log_probs(self, texts, context=""):
self.calls += len(texts)
self.on_call()
return [0.0 if self.liked in t else -50.0 for t in texts]
class RerankerTests(unittest.TestCase):
def engine(self, reranker):
e = make_engine()
e.reranker = reranker
return e
def test_rerank_picks_what_the_model_likes(self):
from autozhuyin.core.reranker import Reranker, ST_ACTIVE
self.assertEqual(type_keys("wu0 \n", self.engine(None)), "天")
r = Reranker("standard", FakeLM("添"))
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "添")
self.assertEqual(r.status, ST_ACTIVE)
def test_off_and_loading_use_the_lexicon(self):
from autozhuyin.core.reranker import Reranker, ST_LOADING, ST_OFF
off = Reranker("off", FakeLM("添"))
self.assertEqual(type_keys("wu0 \n", self.engine(off)), "天")
self.assertEqual(off.status, ST_OFF)
loading = Reranker("standard", None)
self.assertEqual(type_keys("wu0 \n", self.engine(loading)), "天")
self.assertEqual(loading.status, ST_LOADING)
def test_paused_and_reduced(self):
from autozhuyin.core.reranker import PAUSED, REDUCED, Reranker, ST_BUSY
r = Reranker("standard", FakeLM("添"))
r.system_state = lambda: PAUSED
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "天")
self.assertEqual(r.status, ST_BUSY)
lm = FakeLM("添")
r = Reranker("quality", lm)
r.system_state = lambda: REDUCED
type_keys("wu0 \n", self.engine(r))
self.assertLessEqual(lm.calls, 3)
def test_slow_model_is_paused(self):
from autozhuyin.core.reranker import Reranker, ST_SLOW
clock = [0.0]
lm = FakeLM("添")
lm.on_call = lambda: clock.__setitem__(0, clock[0] + 1) # every batch takes a "second"
r = Reranker("standard", lm, now=lambda: clock[0])
for _ in range(3):
type_keys("wu0 \n", self.engine(r))
self.assertEqual(r.status, ST_SLOW)
lm.on_call = lambda: None
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "天") # paused
clock[0] += 61
self.assertEqual(type_keys("wu0 \n", self.engine(r)), "添") # back after a minute
def test_kbest_first_is_viterbi(self):
syl = [s.syllable for s in bopomofo.parse(list("rup wu0 wu0 fu4")).syllables]
conv = Converter(LEX)
best = conv.kbest(syl, 5)
self.assertEqual("".join(t for _, t in best[0][1]), "".join(t for _, t in conv.convert(syl)))
self.assertEqual(len(best), 5)
class RealModelTests(unittest.TestCase):
"""Needs numpy and data/lm.bin (scripts/fetch-lm.sh, tools/build_data.py); skipped otherwise."""
def test_fixes_homophones(self):
from autozhuyin.core import lm
from autozhuyin.core.reranker import Reranker
path = os.path.join(DATA, "lm.bin")
if not lm.available() or not os.path.exists(path):
self.skipTest("needs numpy and the model")
model = lm.GPT2LanguageModel(path, os.path.join(DATA, "lm-vocab.txt"))
self.assertAlmostEqual(model.log_prob("他在家"), -14.329, delta=0.05)
cases = {"w8 ql32k71u3ji3dj94cp32ji ": "他跑得比我快很多", "fu/3su3y94gji u h4cl3a87": "請你再說一次好嗎"}
for keys, want in cases.items():
self.assertNotEqual(type_keys(keys + "\n", make_engine()), want)
e = make_engine()
e.reranker = Reranker("standard", model)
self.assertEqual(type_keys(keys + "\n", e), want)
class AISettingsTests(unittest.TestCase):
def test_settings_round_trip_and_status(self):
import tempfile
from autozhuyin.core.ai import AI
user = tempfile.mkdtemp()
ai = AI(tempfile.mkdtemp(), user, background=False) # no model files here
self.assertTrue(ai.error) # "需要 numpy" or "AI 模型未安裝"
self.assertEqual(ai.status_text(), ai.error)
ai.set_mode("saver")
ai.set_auto_pause(False)
again = AI(tempfile.mkdtemp(), user, background=False)
self.assertEqual(again.reranker.mode, "saver")
self.assertFalse(again.auto_pause)
if __name__ == "__main__":
unittest.main()
+33
View File
@@ -40,6 +40,9 @@ class FakeTextService:
def setCandidateCursor(self, p): self.reply["candidateCursor"] = p
def showMessage(self, message, duration=3): self.reply["showMessage"] = message
def addButton(self, button_id, **kw): self.reply.setdefault("addButton", []).append(button_id)
def removeButton(self, button_id): self.reply.setdefault("removeButton", []).append(button_id)
def changeButton(self, button_id, **kw): self.reply.setdefault("changeButton", []).append((button_id, kw))
def setShowCandidates(self, show):
self.showCandidates = show
@@ -47,6 +50,7 @@ class FakeTextService:
textService.TextService = FakeTextService
textService.COMMAND_LEFT_CLICK, textService.COMMAND_RIGHT_CLICK, textService.COMMAND_MENU = 0, 1, 2
sys.modules["textService"] = textService
from autozhuyin.autozhuyin_ime import AutoZhuyinTextService, _shared_classifier # noqa: E402
@@ -114,6 +118,29 @@ class WrapperTests(unittest.TestCase):
self.assertEqual(type_text(self.svc, "`<"), ",")
self.assertEqual(type_text(self.svc, "ji3`."), "我。")
def test_settings_menu(self):
from autozhuyin import autozhuyin_ime as ime
self.assertIn("autozhuyin-settings", self.svc.reply["addButton"])
menu = self.svc.onMenu("autozhuyin-settings")
texts = [m.get("text") for m in menu]
self.assertIn("AI 選字:標準", texts)
self.assertIn("清除學習紀錄", texts)
self.svc.onCommand(ime.ID_AI_MODE["saver"], 2)
checked = [m["text"] for m in self.svc.onMenu("autozhuyin-settings") if m.get("checked")]
self.assertIn("AI 選字:省電", checked)
self.svc.onCommand(ime.ID_AI_MODE["standard"], 2)
self.assertIsNone(self.svc.onMenu("other"))
def test_mode_icon_click_toggles_english(self):
self.svc.reply = {}
self.svc.onCommand(1, 0)
self.assertTrue(self.svc.engine.english_mode)
(button, kw), = self.svc.reply["changeButton"]
self.assertEqual(button, "windows-mode-icon")
self.assertTrue(kw["icon"].endswith("icon-en.ico") or kw["icon"].endswith("icon.ico"))
self.svc.onCommand(1, 0)
self.assertFalse(self.svc.engine.english_mode)
def test_composition_shown(self):
type_text(self.svc, "su3c")
self.assertEqual(self.svc.reply["compositionString"], "你ㄏ")
@@ -152,6 +179,12 @@ class ShiftTapWrapperTests(unittest.TestCase):
self.svc = AutoZhuyinTextService(client=None)
self.svc.onActivate()
def test_shift_tap_changes_icon(self):
tap_shift(self.svc)
self.assertEqual(self.svc.reply["changeButton"][0][0], "windows-mode-icon")
tap_shift(self.svc)
self.assertIn("changeButton", self.svc.reply)
def test_tap_forces_english_lowercase(self):
reply = tap_shift(self.svc)
self.assertEqual(reply.get("showMessage"), "英文")
+7
View File
@@ -5,6 +5,7 @@
- english-dict.txt: filtered from a system word list (default /usr/share/dict/words,
Webster's 2nd edition, public domain), since Windows has no such file
- english-bigram.json: precomputed letter-bigram table
- lm.bin, lm-vocab.txt: the "AI 選字" language model, copied from build/lm/ if present
Usage: build_data.py [word-list]
"""
@@ -34,4 +35,10 @@ common = {w.strip().lower() for w in (data / "english-common.txt").read_text(enc
bigram = train_bigram(sorted(common | set(words)), common)
(data / "english-bigram.json").write_text(json.dumps(bigram), encoding="utf-8")
# "AI 選字" language model (scripts/fetch-lm.sh makes build/lm/); optional.
lm_dir = ROOT / "build" / "lm"
for name in ("lm.bin", "lm-vocab.txt"):
if (lm_dir / name).exists():
shutil.copy(lm_dir / name, data / name)
print("data ready: %d dictionary words, %d bigrams" % (len(words), len(bigram["table"])))
+79
View File
@@ -0,0 +1,79 @@
"""Downloads numpy for one Python version / platform from PyPI and unpacks it into
<dest>/<python tag>/ (numpy, numpy.libs, dist-info; test folders are dropped).
fetch_numpy.py <dest> <python tag> <platform>
fetch_numpy.py autozhuyin/vendor cp38 win32
fetch_numpy.py ~/.local/share/autozhuyin/vendor auto auto (this Python, Linux)
The newest numpy release that has a wheel for that combination is used, and its
SHA-256 is checked against PyPI. Standard library only.
"""
import hashlib
import io
import json
import platform
import re
import shutil
import sys
import urllib.request
import zipfile
from pathlib import Path
def version_key(v):
return tuple(int(x) for x in re.findall(r"\d+", v))
def linux_platforms():
machine = platform.machine().lower()
arch = {"amd64": "x86_64", "arm64": "aarch64"}.get(machine, machine)
return [f"manylinux_2_17_{arch}", f"manylinux2014_{arch}", f"manylinux_2_27_{arch}", f"manylinux_2_28_{arch}"]
def find_wheel(tag, platforms):
with urllib.request.urlopen("https://pypi.org/pypi/numpy/json", timeout=60) as r:
releases = json.load(r)["releases"]
for version in sorted(releases, key=version_key, reverse=True):
if re.search(r"[a-z]", version): # skip pre-releases (rc, b, a, dev)
continue
for f in releases[version]:
name = f["filename"]
if not name.endswith(".whl") or f.get("yanked"):
continue
if f"-{tag}-{tag}-" not in name: # also skips free-threaded "cp315t" builds
continue
plat = name[:-4].split("-")[-1]
if any(p in plat.split(".") for p in platforms):
return version, f
return None, None
def main(dest, tag, plat):
if tag == "auto":
tag = "cp%d%d" % sys.version_info[:2]
platforms = linux_platforms() if plat == "auto" else [plat]
version, f = find_wheel(tag, platforms)
if not f:
sys.exit(f"no numpy wheel for {tag} {platforms}")
print(f"numpy {version} for {tag}: {f['filename']} ({f['size'] / 1e6:.1f} MB)")
with urllib.request.urlopen(f["url"], timeout=300) as r:
data = r.read()
if hashlib.sha256(data).hexdigest() != f["digests"]["sha256"]:
sys.exit("checksum mismatch")
out = Path(dest).expanduser() / tag
if out.exists():
shutil.rmtree(out)
out.mkdir(parents=True)
with zipfile.ZipFile(io.BytesIO(data)) as z:
for info in z.infolist():
parts = Path(info.filename).parts
if "tests" in parts or parts[0].endswith(".data"):
continue
z.extract(info, out)
print(f"unpacked to {out}")
if __name__ == "__main__":
if len(sys.argv) != 4:
sys.exit(__doc__)
main(*sys.argv[1:])