Compare commits

..
13 Commits
Author SHA1 Message Date
ays953 bb9cb0c4e0 Release v0.6.1 2026-11-11 12:00:10 +08:00
ays953andClaude Opus 5.5 2c5c70c066 Learn the previous-word context; fix first-tone characters after English
After an English word, 23 of 50 common first-tone characters (機, 吃, 書…)
were read as English when space was pressed: the fixed +1.5 context bonus
was far too strong for the trained weight scale, and the training data never
contained the "space just pressed on a first-tone syllable" decision.

Context is now the ctx.zh / ctx.lit features, trained with the rest; the
synthetic data includes the previous mode and the first-tone space decision.
Test accuracy: commit 98.22% -> 99.52%, typing 88.73% -> 91.49%, and when the
language differs from the previous word 92.0% -> 99.3%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:59:57 +08:00
ays953 17f7d6f50d Release v0.6.0 2026-11-11 11:44:38 +08:00
ays953andClaude Opus 5.5 7e173a1da8 Make the Shift tap a persistent English-mode toggle
A Shift tap with nothing composed now switches English mode on until the
next tap, instead of forcing only the next word: space, Enter and other keys
no longer drop back to automatic detection. In English mode every key goes
straight to the app, so there is no composition or underline. A Shift tap
while typing still flips the buffer like Tab. Windows shows "英文" /
"中英自動" when toggling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:44:25 +08:00
ays953 fa0b78e5a4 Release v0.5.1 2026-11-11 11:36:12 +08:00
ays953andClaude Opus 5.5 e928d08fdc Type forced English directly, without an underlined composition
After a Shift tap, letters, digits and word punctuation go straight to the
app instead of into marked text, so nothing is underlined. Space, Enter, Tab,
Escape or another symbol ends the forced word and automatic detection
resumes; backspace is handled by the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:35:59 +08:00
ays953 82f20a6551 Release v0.5.0 2026-11-11 10:37:37 +08:00
ays953andClaude Opus 5.5 e6e5004630 Add a composition cursor so candidates can be chosen for any character
Left/Right move a cursor between Zhuyin syllables in the composition; Down
opens candidates for the phrase ending at the cursor instead of always the
last syllable. Typing or backspace returns the cursor to the end. Candidates
also work for Chinese that is followed by English in the same buffer. The Mac
caret and PIME composition cursor follow it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:37:24 +08:00
ays953 988ed471be Release v0.4.1 2026-11-11 10:18:14 +08:00
ays953andClaude Opus 5.5 6c36c01f0e Stop declaring Latin repertoire so password fields bypass the IME
With "Latn" in tsInputMethodCharacterRepertoireKey macOS treated AutoZhuyin
as ASCII-capable and kept it active in secure text fields, so typed passwords
showed up as underlined composition text. Apple's Zhuyin and vChewing declare
only Hant (not ASCII-capable); macOS then switches to an ASCII keyboard layout
in password fields and the input method never sees the keystrokes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:18:01 +08:00
ays953andClaude Opus 5.5 0fc42a4b8c install.sh: warn when a system-wide .pkg copy shadows the per-user install
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:05:24 +08:00
ays953 42a2d3a2a3 Release v0.4.0 2026-11-11 09:57:34 +08:00
ays953andClaude Opus 5.5 20291bf515 Add Shift tap to force English for the next word
Pressing and releasing Shift on its own arms literal mode for the next word,
so lowercase English can be typed without it being read as Zhuyin; it resets
after the word is committed and a second tap cancels it. While typing, a
Shift tap flips the buffer like Tab. Forcing before typing is not recorded
as a learned correction; flipping is.

Mac: listens for flagsChanged events. Windows: Shift key-up via PIME's
filterKeyUp/onKeyUp, with a short "英文" tooltip. The tap detection lives in
the core (ShiftTapDetector / shift_tap.py) with tests on both sides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:57:18 +08:00
23 changed files with 5834 additions and 3514 deletions
+11 -8
View File
@@ -12,9 +12,10 @@
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
- **候選字**:按 ↓ 選最後一個字或詞,選過的會固定下來
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`
- **完整詞庫**:約 17 萬筆,來自小麥注音 McBopomofo(MIT 授權)
@@ -50,9 +51,11 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
| 空白 | 注音:一聲;音節已完成時送出中文。英文:送出單字加空白 |
| Enter | 送出目前組字 |
| **Tab** | 判斷錯了?按 Tab 把整段翻轉成另一種語言,輸入法會**記住**這次修正 |
| ↓ | 開啟候選字(選最後一個字或詞),數字鍵 1–9 選字,Esc 關閉 |
| ← → | 在組字區移動游標,選要改哪個字 |
| ↓ | 開啟游標前那個字或詞的候選字(游標預設在最後),數字鍵 1–9 選字,Esc 關閉 |
| Esc | 取消組字 |
| Shift + 字母 | 強制英文 |
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷。打到一半時:同 Tab,整段翻轉 |
| Shift + 字母 | 打大寫字母(也會強制英文) |
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
@@ -141,12 +144,12 @@ python3 training/make_parity.py # 權重或特徵改了之後都要重
目前的結果(10,000 筆測試資料):
| | 送出時整段判斷正確 | 打字過程中顯示正確 |
|---|---|---|
| 手調權重 | 97.58% | 86.15% |
| 訓練後 | **99.62%** | 87.81% |
| | 送出時整段判斷正確 | 打字過程中顯示正確 | 跟前一個字語言不同時 |
|---|---|---|---|
| 手調權重 | 97.84% | 88.19% | 96.2% |
| 訓練後 | **99.52%** | **91.49%** | **99.3%** |
送出時的錯誤率從 2.4% 降到 0.4%。「打字過程中」本來就有歧義(例如 `ji3h` 的 `h` 可能是英文的開頭,也可能是注音 ㄘ),所以準確率比較低,但最後送出的結果才是重點。
訓練資料也模擬了「前一個字是中文或英文」和「按下空白(一聲)的那一刻」,所以「前一個字」的影響(`ctx.zh` / `ctx.lit`)也是訓練出來的,不是手調的。「打字過程中」本來就有歧義(例如 `ji3h` 的 `h` 可能是英文的開頭,也可能是注音 ㄘ),所以準確率比較低,但最後送出的結果才是重點。
新增特徵時,Swift 和 Python 兩邊都要加,預設權重設成 0,然後重新訓練。
+2 -3
View File
@@ -15,9 +15,9 @@
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>0.3.0</string>
<string>0.6.1</string>
<key>CFBundleVersion</key>
<string>3</string>
<string>9</string>
<key>InputMethodConnectionName</key>
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
<key>InputMethodServerControllerClass</key>
@@ -37,7 +37,6 @@
<key>tsInputMethodCharacterRepertoireKey</key>
<array>
<string>Hant</string>
<string>Latn</string>
</array>
<key>tsInputMethodIconFileKey</key>
<string>icon.tiff</string>
+28 -26
View File
@@ -1,39 +1,41 @@
{
"about": "AutoZhuyin classifier weights, learned by training/train.py",
"trained": "2026-11-11",
"examples": 40001,
"examples": 40000,
"test_accuracy": {
"commit": 0.9962,
"typing": 0.8781
"commit": 0.9952,
"typing": 0.9149
},
"weights": {
"lit.bias": 0.4276,
"lit.common": -0.3465,
"lit.common_len": 0.81,
"lit.dict": -1.7902,
"lit.dict_len": 0.6765,
"lit.digits": -2.7403,
"lit.num_digits": -0.0751,
"lit.num_others": -0.7433,
"lit.others": -1.5685,
"lit.prefix_common_len": 0.2082,
"lit.prefix_dict_len": 0.0003,
"lit.short": -0.2397,
"lit.unknown_score": 0.213,
"ctx.lit": 0.0916,
"ctx.zh": 0.1349,
"lit.bias": 0.3982,
"lit.common": -0.2906,
"lit.common_len": 0.8046,
"lit.dict": -1.6662,
"lit.dict_len": 0.6441,
"lit.digits": -2.9689,
"lit.num_digits": -0.018,
"lit.num_others": -0.5554,
"lit.others": -2.0002,
"lit.prefix_common_len": 0.2831,
"lit.prefix_dict_len": 0.0299,
"lit.short": -0.2482,
"lit.unknown_score": 0.2479,
"lit.upper": 20.0,
"seg.switch": -1.7203,
"zh.bias": 1.3522,
"zh.lexscore": 0.26,
"seg.switch": -1.7596,
"zh.bias": 1.3423,
"zh.lexscore": 0.2917,
"zh.pending_invalid": -5.0,
"zh.pending_prefix": -0.9295,
"zh.phrase": 0.4595,
"zh.reordered": -1.4348,
"zh.pending_prefix": -0.9162,
"zh.phrase": 0.4499,
"zh.reordered": -1.3079,
"zh.stray": -5.0,
"zh.toned_invalid": -5.0,
"zh.toned_valid_lex": -0.9823,
"zh.toned_valid_nolex": -1.8742,
"zh.unconfirmed": -0.7891,
"zh.toned_valid_lex": -0.9018,
"zh.toned_valid_nolex": -1.6341,
"zh.unconfirmed": -0.7952,
"zh.untoned_mid_invalid": -5.0,
"zh.untoned_mid_valid": -2.8009
"zh.untoned_mid_valid": -2.7479
}
}
+22 -3
View File
@@ -34,15 +34,32 @@ enum Shared {
@objc(AutoZhuyinInputController)
final class InputController: IMKInputController {
private let engine = CompositionEngine(classifier: Shared.classifier)
private var shiftTap = ShiftTapDetector()
// MARK: - Key events
override func recognizedEvents(_ sender: Any!) -> Int {
Int(NSEvent.EventTypeMask.keyDown.rawValue)
Int(NSEvent.EventTypeMask.keyDown.rawValue | NSEvent.EventTypeMask.flagsChanged.rawValue)
}
override func handle(_ event: NSEvent!, client sender: Any!) -> Bool {
guard let event, event.type == .keyDown, let client = sender as? IMKTextInput else { return false }
guard let event, let client = sender as? IMKTextInput else { return false }
// Shift pressed and released alone: next word is English (or flip the current one).
if event.type == .flagsChanged {
let flags = event.modifierFlags
let tapped = shiftTap.modifiersChanged(
shift: flags.contains(.shift),
otherModifiers: !flags.intersection([.command, .control, .option, .function]).isEmpty,
time: event.timestamp)
if tapped {
_ = engine.handle(.shiftTap)
updateComposition(client)
}
return false
}
guard event.type == .keyDown else { return false }
shiftTap.keyPressed()
// Shortcuts (⌘C, ⌃A, ⌥…) go to the app; commit any composition first.
if !event.modifierFlags.intersection([.command, .control, .option]).isEmpty {
@@ -100,8 +117,10 @@ final class InputController: IMKInputController {
]))
}
let text = marked.string
// Caret at the candidate cursor (UTF-16 offset of the first `cursorOffset` characters).
let caret = String(text.prefix(engine.cursorOffset)).utf16.count
client.setMarkedText(marked,
selectionRange: NSRange(location: (text as NSString).length, length: 0),
selectionRange: NSRange(location: caret, length: 0),
replacementRange: NSRange(location: NSNotFound, length: 0))
if engine.isShowingCandidates {
+10 -14
View File
@@ -58,9 +58,6 @@ public final class Classifier {
/// Feature weights: the hand-tuned defaults overlaid with trained ones.
public private(set) var weights: FeatureVector
/// How much the previous segment's mode pulls the current one.
public var contextWeight = 1.5
public init(lexicon: Lexicon, english: EnglishModel, learned: LearnedPreferences = LearnedPreferences(),
weights: FeatureVector = [:]) {
self.lexicon = lexicon
@@ -69,23 +66,22 @@ public final class Classifier {
self.weights = Features.defaultWeights.merging(weights) { _, new in new }
}
/// (Zhuyin features or nil, literal features or nil) for one segment.
public func segmentFeatures(_ seg: [Character], isLast: Bool, afterOther: Bool,
finishing: Bool) -> (FeatureVector?, FeatureVector?) {
(Features.zhuyin(seg, lexicon: lexicon, isLast: isLast, afterOther: afterOther, finishing: finishing),
Features.literal(seg, english: english, isLast: isLast, afterOther: afterOther, finishing: finishing))
/// (Zhuyin features or nil, literal features or nil) for one segment. `previous` (mode of
/// the text committed just before) only affects the first segment.
public func segmentFeatures(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
previous: InputMode? = nil) -> (FeatureVector?, FeatureVector?) {
(Features.zhuyin(seg, lexicon: lexicon, isLast: isLast, afterOther: afterOther,
finishing: finishing, previous: previous),
Features.literal(seg, english: english, isLast: isLast, afterOther: afterOther,
finishing: finishing, previous: previous))
}
private func scores(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
previous: InputMode?, whole: Bool) -> [Double] {
let (fz, fl) = segmentFeatures(seg, isLast: isLast, afterOther: afterOther, finishing: finishing)
let (fz, fl) = segmentFeatures(seg, isLast: isLast, afterOther: afterOther,
finishing: finishing, previous: previous)
var zh = fz.map { Features.dot(weights, $0) } ?? -.infinity
var lit = fl.map { Features.dot(weights, $0) } ?? -.infinity
switch previous {
case .zhuyin?: zh += contextWeight
case .literal?: lit += contextWeight
case nil: break
}
if whole {
let b = learned.bias(for: String(seg))
if b > 0 { zh += b } else { lit -= b }
+105 -18
View File
@@ -4,6 +4,8 @@ public enum KeyInput: Equatable {
case character(Character)
case backspace, enter, escape, tab
case up, down, left, right
/// Shift pressed and released on its own.
case shiftTap
}
public struct KeyResult: Equatable {
@@ -33,12 +35,19 @@ public final class CompositionEngine {
public let converter: Converter
public private(set) var keys: [Character] = []
/// Set by Tab: overrides the classifier for the whole buffer.
/// Set by Tab / a Shift tap while typing: overrides the classifier for the whole
/// buffer. It is a correction, learned on commit.
public private(set) var forcedMode: InputMode?
/// English mode, toggled by a Shift tap when nothing is being typed. Keys go straight
/// to the app (no composition, no underline) until Shift is tapped again.
public private(set) var isEnglishMode = false
/// User-picked phrases: Zhuyin segment start (key index) → syllable range → text.
private var fixed: [Int: [Range<Int>: String]] = [:]
public private(set) var candidates: [Candidate] = []
private var candidateSegmentStart = 0
/// Composition cursor for choosing candidates: index into `cursorStops`, nil = at the end.
/// ← → move it; ↓ shows candidates for the phrase ending at the cursor.
private var cursor: Int?
public var candidateIndex = 0
public var isShowingCandidates: Bool { !candidates.isEmpty }
@@ -83,17 +92,64 @@ public final class CompositionEngine {
/// Text for one segment. A non-final Zhuyin segment shows its unfinished syllable as Bopomofo.
private func render(_ seg: Segment, final: Bool) -> String {
let segKeys = Array(keys[seg.range])
guard seg.mode == .zhuyin else { return String(segKeys) }
let p = ZhuyinParser.parse(segKeys)
let fixedHere = (fixed[seg.range.lowerBound] ?? [:])
if final {
let all = p.syllables.map(\.syllable)
return converter.convert(all, fixed: fixedHere.filter { $0.key.upperBound <= all.count }).map(\.text).joined()
guard seg.mode == .zhuyin else { return String(keys[seg.range]) }
let (converted, pending) = conversion(of: seg, final: final)
return converted.map(\.text).joined() + pending
}
/// Syllables of a Zhuyin segment that are converted to Chinese (all of them when
/// final, otherwise the finished ones).
private func convertibleSyllables(_ seg: Segment, final: Bool) -> (syllables: [Syllable], pending: String) {
let p = ZhuyinParser.parse(Array(keys[seg.range]))
if final { return (p.syllables.map(\.syllable), "") }
return (p.completed.map(\.syllable), p.pending?.syllable.reading ?? "")
}
private func conversion(of seg: Segment, final: Bool) -> (segments: [ConvertedSegment], pending: String) {
let (syl, pending) = convertibleSyllables(seg, final: final)
let fixedHere = (fixed[seg.range.lowerBound] ?? [:]).filter { $0.key.upperBound <= syl.count }
return (converter.convert(syl, fixed: fixedHere), pending)
}
// MARK: - Cursor
/// A place the cursor can stop: after the `local`-th syllable of Zhuyin segment `segIndex`.
private struct CursorStop { let segIndex: Int; let local: Int }
private var cursorStops: [CursorStop] {
let segs = segments
var out: [CursorStop] = []
for (i, seg) in segs.enumerated() where seg.mode == .zhuyin {
let n = convertibleSyllables(seg, final: i < segs.count - 1).syllables.count
for k in stride(from: 1, through: n, by: 1) { out.append(CursorStop(segIndex: i, local: k)) }
}
let done = p.completed.map(\.syllable)
let text = converter.convert(done, fixed: fixedHere.filter { $0.key.upperBound <= done.count }).map(\.text).joined()
return text + (p.pending?.syllable.reading ?? "")
return out
}
/// Cursor position in `markedText`, in Characters.
public var cursorOffset: Int {
let pieces = markedPieces
let total = pieces.reduce(0) { $0 + $1.text.count }
let stops = cursorStops
guard let c = cursor, stops.indices.contains(c) else { return total }
let stop = stops[c]
var offset = pieces[..<stop.segIndex].reduce(0) { $0 + $1.text.count }
let segs = segments
for conv in conversion(of: segs[stop.segIndex], final: stop.segIndex < segs.count - 1).segments {
if conv.range.upperBound <= stop.local {
offset += conv.text.count
} else if conv.range.lowerBound < stop.local {
offset += min(stop.local - conv.range.lowerBound, conv.text.count)
}
}
return offset
}
private func moveCursor(by delta: Int) {
let stops = cursorStops
guard !stops.isEmpty else { return }
let next = (cursor ?? stops.count - 1) + delta
cursor = next >= stops.count - 1 ? nil : max(0, next)
}
// MARK: - Key handling
@@ -106,6 +162,7 @@ public final class CompositionEngine {
return handleCharacter(c)
case .backspace:
guard !keys.isEmpty else { return .passThrough }
cursor = nil
keys.removeLast()
fixed = fixed.filter { $0.key < keys.count }
if keys.isEmpty { forcedMode = nil; fixed = [:] }
@@ -119,18 +176,44 @@ public final class CompositionEngine {
return .consumed
case .tab:
guard !keys.isEmpty else { return .passThrough }
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
flip()
return .consumed
case .shiftTap:
if keys.isEmpty {
// Toggle English mode; it stays on until the next Shift tap.
isEnglishMode.toggle()
previousMode = isEnglishMode ? .literal : nil
previousCommitTime = now()
} else {
flip()
}
return .consumed
case .down:
guard !keys.isEmpty else { return .passThrough }
openCandidates()
return .consumed
case .up, .left, .right:
case .left:
guard !keys.isEmpty else { return .passThrough }
moveCursor(by: -1)
return .consumed
case .right:
guard !keys.isEmpty else { return .passThrough }
moveCursor(by: 1)
return .consumed
case .up:
return keys.isEmpty ? .passThrough : .consumed
}
}
/// Forces the whole buffer into the other mode (a correction that is learned).
private func flip() {
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
}
private func handleCharacter(_ c: Character) -> KeyResult {
cursor = nil // typing always continues at the end
// English mode: every key goes to the app as typed.
if isEnglishMode && keys.isEmpty { return .passThrough }
if c == " " {
guard !keys.isEmpty else { return .passThrough }
let segs = segments(of: keys, finishing: true)
@@ -178,13 +261,16 @@ public final class CompositionEngine {
// MARK: - Candidates (for the last Zhuyin segment)
/// Opens candidates for the phrase ending at the cursor (default: the last syllable).
private func openCandidates() {
let stops = cursorStops
guard !stops.isEmpty else { return }
let stop = stops[min(cursor ?? stops.count - 1, stops.count - 1)]
let segs = segments
guard let last = segs.last, last.mode == .zhuyin else { return }
let done = ZhuyinParser.parse(Array(keys[last.range])).completed.map(\.syllable)
guard !done.isEmpty else { return }
candidates = converter.candidates(done, endingAt: done.count)
candidateSegmentStart = last.range.lowerBound
let seg = segs[stop.segIndex]
let syl = convertibleSyllables(seg, final: stop.segIndex < segs.count - 1).syllables
candidates = converter.candidates(syl, endingAt: stop.local)
candidateSegmentStart = seg.range.lowerBound
candidateIndex = 0
}
@@ -240,6 +326,7 @@ public final class CompositionEngine {
public func reset() {
keys = []
cursor = nil
forcedMode = nil
fixed = [:]
candidates = []
+17 -4
View File
@@ -42,6 +42,9 @@ public enum Features {
"lit.others": -1.5,
// Segmentation
"seg.switch": -3.0,
// Context: the first segment has the same mode as the previously committed text
"ctx.zh": 1.5,
"ctx.lit": 1.5,
]
/// Reads {"weights": {...}} from a JSON file; empty if missing or invalid.
@@ -58,7 +61,8 @@ public enum Features {
/// Features of `keys` read as Zhuyin, or nil if impossible.
public static func zhuyin(_ keys: [Character], lexicon: Lexicon, isLast: Bool = true,
afterOther: Bool = false, finishing: Bool = false) -> FeatureVector? {
afterOther: Bool = false, finishing: Bool = false,
previous: InputMode? = nil) -> FeatureVector? {
let parse = ZhuyinParser.parse(keys)
// A Zhuyin segment followed by another must end with a tone.
if !isLast && parse.pending != nil { return nil }
@@ -107,18 +111,27 @@ public enum Features {
if isLast && afterOther && !finishing && !parse.syllables.contains(where: \.toned) {
add("zh.unconfirmed")
}
if afterOther { add("seg.switch") }
if afterOther {
add("seg.switch")
} else if previous == .zhuyin {
add("ctx.zh")
}
return f
}
/// Features of `keys` output as typed, or nil if impossible.
public static func literal(_ keys: [Character], english: EnglishModel, isLast: Bool = true,
afterOther: Bool = false, finishing: Bool = false) -> FeatureVector? {
afterOther: Bool = false, finishing: Bool = false,
previous: InputMode? = nil) -> FeatureVector? {
if keys.isEmpty || keys.contains(" ") { return nil }
let complete = !isLast || finishing
var f: FeatureVector = ["lit.bias": 1]
func add(_ k: String, _ v: Double = 1) { f[k, default: 0] += v }
if afterOther { add("seg.switch") }
if afterOther {
add("seg.switch")
} else if previous == .literal {
add("ctx.lit")
}
if keys.contains(where: { $0.isUppercase }) { add("lit.upper") }
let letters = String(keys.filter { $0.isLetter })
@@ -0,0 +1,32 @@
import Foundation
/// Detects Shift pressed and released on its own (no other key in between, released
/// within `maxDuration`), as opposed to Shift used for capitals or shortcuts.
///
/// Python twin: windows/autozhuyin/core/shift_tap.py
public struct ShiftTapDetector {
public var maxDuration: TimeInterval = 0.5
private var pressedAt: TimeInterval?
private var used = false
public init() {}
/// Call when the modifier state changes. Returns true when a Shift tap completed.
public mutating func modifiersChanged(shift: Bool, otherModifiers: Bool, time: TimeInterval) -> Bool {
if shift && !otherModifiers {
if pressedAt == nil {
pressedAt = time
used = false
}
return false
}
defer { pressedAt = nil }
guard let start = pressedAt, !shift, !otherModifiers, !used else { return false }
return time - start <= maxDuration
}
/// Call for every other key press: Shift was used as a modifier, not tapped.
public mutating func keyPressed() {
if pressedAt != nil { used = true }
}
}
@@ -265,10 +265,13 @@ final class ParityTests: XCTestCase {
for fx in fixtures {
let keys = Array(fx["keys"] as! String)
let expected = (fx["segments"] as! [[Any]]).map { "\($0[0])-\($0[1]):\($0[2])" }
let got = c.segment(keys, finishing: fx["finishing"] as! Bool).map {
let previous = (fx["previous"] as? String).flatMap(InputMode.init(rawValue:))
let got = c.segment(keys, finishing: fx["finishing"] as! Bool, previous: previous).map {
"\($0.range.lowerBound)-\($0.range.upperBound):\($0.mode.rawValue)"
}
if got != expected { mismatches.append("\(String(keys)) finishing=\(fx["finishing"]!): \(got) vs \(expected)") }
if got != expected {
mismatches.append("\(String(keys)) finishing=\(fx["finishing"]!) previous=\(String(describing: previous)): \(got) vs \(expected)")
}
}
XCTAssertGreaterThan(fixtures.count, 300)
XCTAssertTrue(mismatches.isEmpty, "\(mismatches.count) mismatches:\n" + mismatches.prefix(10).joined(separator: "\n"))
@@ -281,3 +284,143 @@ final class ParityTests: XCTestCase {
XCTAssertEqual(out.count, 1, out)
}
}
final class ShiftTapTests: XCTestCase {
func engine() -> CompositionEngine {
CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
}
func type(_ s: String, into e: CompositionEngine) -> String {
var out = ""
for c in s {
let key: KeyInput = c == "⇧" ? .shiftTap : c == "\n" ? .enter : .character(c)
let r = e.handle(key)
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
}
return out
}
func testShiftTapTogglesEnglishMode() {
let e = engine()
// "ru" alone would be ㄐㄧ (機); in English mode keys pass straight through, lowercase.
XCTAssertEqual(type("⇧ru su ", into: e), "ru su ")
XCTAssertTrue(e.isEnglishMode) // still English after spaces
XCTAssertEqual(e.markedText, "") // never composed: no underline
XCTAssertEqual(type("\n", into: e), "\n") // Enter goes to the app, mode stays
XCTAssertTrue(e.isEnglishMode)
XCTAssertEqual(type("⇧su3cl3\n", into: e), "你好") // tap again: automatic again
XCTAssertFalse(e.isEnglishMode)
}
func testShiftTapTwiceCancels() {
XCTAssertEqual(type("⇧⇧su3cl3\n", into: engine()), "你好")
}
func testShiftTapMidWordFlips() {
let e = engine()
_ = type("so", into: e)
XCTAssertEqual(e.mode, .literal)
_ = e.handle(.shiftTap)
XCTAssertEqual(e.mode, .zhuyin)
XCTAssertFalse(e.isEnglishMode) // a flip, not English mode
_ = type("\n", into: e)
XCTAssertEqual(e.classifier.learned.weights["so"], 1) // a correction is learned
}
func testEnglishModeIsNotLearned() {
let e = engine()
_ = type("⇧hello ", into: e)
XCTAssertTrue(e.classifier.learned.weights.isEmpty)
}
func testBackspaceInEnglishModeGoesToApp() {
let e = engine()
_ = type("⇧", into: e)
XCTAssertFalse(e.handle(.backspace).handled)
XCTAssertTrue(e.isEnglishMode)
}
func testDetector() {
var d = ShiftTapDetector()
XCTAssertFalse(d.modifiersChanged(shift: true, otherModifiers: false, time: 0))
XCTAssertTrue(d.modifiersChanged(shift: false, otherModifiers: false, time: 0.1))
// Shift used for a capital letter: not a tap.
_ = d.modifiersChanged(shift: true, otherModifiers: false, time: 1)
d.keyPressed()
XCTAssertFalse(d.modifiersChanged(shift: false, otherModifiers: false, time: 1.1))
// Held too long: not a tap.
_ = d.modifiersChanged(shift: true, otherModifiers: false, time: 2)
XCTAssertFalse(d.modifiersChanged(shift: false, otherModifiers: false, time: 3))
// Shift + Command: not a tap.
_ = d.modifiersChanged(shift: true, otherModifiers: false, time: 4)
_ = d.modifiersChanged(shift: true, otherModifiers: true, time: 4.1)
XCTAssertFalse(d.modifiersChanged(shift: false, otherModifiers: false, time: 4.2))
}
}
final class CursorTests: XCTestCase {
func engine() -> CompositionEngine {
CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
}
func type(_ s: String, into e: CompositionEngine) -> String {
var out = ""
for c in s {
let key: KeyInput = c == "←" ? .left : c == "→" ? .right : c == "↓" ? .down : c == "\n" ? .enter : .character(c)
let r = e.handle(key)
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
}
return out
}
func testCursorMovesAndClamps() {
let e = engine()
_ = type("rup wu0 wu0 fu4", into: e) // 今天天氣
XCTAssertEqual(e.cursorOffset, 4)
_ = type("←", into: e)
XCTAssertEqual(e.cursorOffset, 3)
_ = type("←←←←", into: e)
XCTAssertEqual(e.cursorOffset, 1) // stops after the first character
_ = type("→→→→", into: e)
XCTAssertEqual(e.cursorOffset, 4)
}
func testChooseCandidateInTheMiddle() {
let e = engine()
_ = type("rup wu0 wu0 fu4", into: e)
_ = type("←←↓", into: e) // cursor after the 2nd syllable (天)
XCTAssertTrue(e.isShowingCandidates)
XCTAssertTrue(e.candidates.allSatisfy { $0.range.upperBound == 2 })
let i = e.candidates.firstIndex { $0.text == "添" }!
e.selectCandidate(i)
XCTAssertEqual(type("\n", into: e), "今添天氣")
}
func testTypingReturnsCursorToEnd() {
let e = engine()
_ = type("rup wu0 ←", into: e)
XCTAssertEqual(e.cursorOffset, 1)
_ = type("wu0 ", into: e)
XCTAssertEqual(e.cursorOffset, e.markedText.count)
}
func testCandidatesForChineseBeforeEnglish() {
let e = engine()
_ = type("su3cl3hello", into: e) // 你好hello
_ = type("↓", into: e) // last Chinese syllable, not the English word
XCTAssertTrue(e.isShowingCandidates)
}
}
final class FirstToneAfterEnglishTests: XCTestCase {
/// After an English word, a first-tone syllable + space must stay Chinese (機, 吃, 書…).
func testFirstToneAfterEnglish() {
for keys in ["ru", "t", "gj", "vup", "tj", "el"] {
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
var out = ""
for c in "hello " + keys + " " { if let t = e.handle(.character(c)).commit { out += t } }
XCTAssertEqual(out, "hello ", keys)
XCTAssertEqual(e.markedText.count, 1, keys) // still composing one Chinese character
}
}
}
+11
View File
@@ -13,6 +13,17 @@ rm -rf "$DEST/AutoZhuyin.app"
cp -R build/AutoZhuyin.app "$DEST/"
"$DEST/AutoZhuyin.app/Contents/MacOS/AutoZhuyin" --register
SYSTEM_APP="/Library/Input Methods/AutoZhuyin.app"
if [ -d "$SYSTEM_APP" ]; then
OLD="$(plutil -extract CFBundleShortVersionString raw "$SYSTEM_APP/Contents/Info.plist" 2>/dev/null || echo '?')"
echo
echo "WARNING: a system-wide copy (v$OLD, installed from the .pkg) exists at:"
echo " $SYSTEM_APP"
echo "macOS runs that copy instead of this one. Install the new .pkg"
echo "(scripts/package.sh), or remove it with:"
echo " sudo rm -rf \"$SYSTEM_APP\" && pkill -x AutoZhuyin"
fi
echo
echo "Installed to $DEST/AutoZhuyin.app"
echo "First install: System Settings → Keyboard → Text Input → Edit… → + → 繁體中文 → 自動注音"
+6 -6
View File
@@ -23,14 +23,14 @@ def main():
clf = core.Classifier(lexicon, english, weights=core.Classifier.load_weights(WEIGHTS_PATH))
gen = Generator(os.path.join(DATA, "lexicon.txt"), os.path.join(DATA, "english-common.txt"),
os.path.join(DATA, "english-dict.txt"), seed=7)
cases = [(k, f) for k in HAND_PICKED for f in (False, True)]
while len(cases) < 400:
cases = [(k, f, p) for k in HAND_PICKED for f in (False, True) for p in (None, "literal")]
while len(cases) < 500:
ex = gen.example()
if len(ex["keys"]) <= 28:
cases.append((ex["keys"], ex["finishing"]))
cases.append((gen.prefix_of(ex)["keys"], False))
fixtures = [{"keys": k, "finishing": f,
"segments": [[s, e, m] for s, e, m in clf.segment(list(k), f)]} for k, f in cases]
cases.append((ex["keys"], ex["finishing"], ex["previous"]))
cases.append((gen.prefix_of(ex)["keys"], False, ex["previous"]))
fixtures = [{"keys": k, "finishing": f, "previous": p,
"segments": [[s, e, m] for s, e, m in clf.segment(list(k), f, p)]} for k, f, p in cases]
with open(OUT, "w", encoding="utf-8") as fh:
json.dump(fixtures, fh, ensure_ascii=False, indent=0)
print("wrote %d fixtures to %s" % (len(fixtures), OUT))
+5114 -3374
View File
File diff suppressed because it is too large Load Diff
+17 -2
View File
@@ -102,16 +102,31 @@ class Generator:
labels = "".join(m * len(p) for m, p in parts)
if parts[-1][0] == "l":
finishing = True # English / numbers end with a space
elif keys[-1] == " " and self.rng.random() < 0.6:
# The moment the space (first tone) is pressed: the buffer has no space yet and
# the input method decides "first tone" vs "English word + space".
keys, labels, finishing = keys[:-1], labels[:-1], True
kind += "/tone1"
else:
# Chinese ends with Enter, or space after a finished (toned) syllable.
finishing = keys[-1] != " " and self.rng.random() < 0.5
return {"keys": keys, "labels": labels, "finishing": finishing, "kind": kind}
return {"keys": keys, "labels": labels, "finishing": finishing, "kind": kind,
"previous": self.previous_mode(labels[0])}
def previous_mode(self, first_label):
"""Mode of the text committed just before: none, mostly the same, sometimes the other."""
r = self.rng.random()
if r < 0.35:
return None
same = core.ZHUYIN if first_label == "z" else core.LITERAL
other = core.LITERAL if first_label == "z" else core.ZHUYIN
return same if r < 0.80 else other
def prefix_of(self, ex):
"""The buffer while still typing: a random cut, not finishing."""
cut = self.rng.randint(1, len(ex["keys"]) - 1) if len(ex["keys"]) > 1 else 1
return {"keys": ex["keys"][:cut], "labels": ex["labels"][:cut], "finishing": False,
"kind": ex["kind"] + "/prefix"}
"kind": ex["kind"] + "/prefix", "previous": ex["previous"]}
def _cumulative(ws):
+19 -10
View File
@@ -45,12 +45,12 @@ def labels_of(segs, n):
return "".join(lab)
def seg_features(clf, keys, segs, finishing):
def seg_features(clf, keys, segs, finishing, previous):
"""Summed features of a segmentation, or None if any segment is impossible."""
total = defaultdict(float)
n = len(keys)
for idx, (s, e, m) in enumerate(segs):
fz, fl = clf.segment_features(keys[s:e], e == n, idx > 0, finishing)
fz, fl = clf.segment_features(keys[s:e], e == n, idx > 0, finishing, previous if idx == 0 else None)
f = fz if m == core.ZHUYIN else fl
if f is None:
return None
@@ -62,12 +62,13 @@ def seg_features(clf, keys, segs, finishing):
def evaluate(clf, weights, data):
by_kind = defaultdict(lambda: [0, 0])
for ex in data:
segs = clf.segment(list(ex["keys"]), ex["finishing"], None, weights)
segs = clf.segment(list(ex["keys"]), ex["finishing"], ex["previous"], weights)
ok = labels_of(segs, len(ex["keys"])) == ex["labels"]
bucket = by_kind["prefix" if ex["kind"].endswith("/prefix") else ex["kind"]]
bucket[0] += ok
bucket[1] += 1
commit = [v for k, v in by_kind.items() if k != "prefix"]
kind = "prefix" if ex["kind"].endswith("/prefix") else ex["kind"]
for k in (kind, "after-other") if _after_other(ex) else (kind,):
by_kind[k][0] += ok
by_kind[k][1] += 1
commit = [v for k, v in by_kind.items() if k not in ("prefix", "after-other")]
summary = {
"commit": sum(c for c, _ in commit) / max(1, sum(t for _, t in commit)),
"prefix": by_kind["prefix"][0] / max(1, by_kind["prefix"][1]),
@@ -76,6 +77,14 @@ def evaluate(clf, weights, data):
return summary
def _after_other(ex):
"""Committed buffer whose first segment differs from the text typed just before."""
if ex["kind"].endswith("/prefix") or ex["previous"] is None:
return False
first = core.ZHUYIN if ex["labels"][0] == "z" else core.LITERAL
return ex["previous"] != first
def show(name, s):
kinds = " ".join("%s %.1f%%" % (k, 100 * v) for k, v in s["by_kind"].items())
print("%-9s commit %.2f%% typing %.2f%% [%s]" % (name, 100 * s["commit"], 100 * s["prefix"], kinds))
@@ -92,11 +101,11 @@ def train(clf, data, epochs, lr, seed):
t0 = time.time()
for ex in data:
keys = list(ex["keys"])
pred = clf.segment(keys, ex["finishing"], None, w)
pred = clf.segment(keys, ex["finishing"], ex["previous"], w)
if labels_of(pred, len(keys)) != ex["labels"]:
gold = gold_segments(ex["labels"])
fg = seg_features(clf, keys, gold, ex["finishing"])
fp = seg_features(clf, keys, pred, ex["finishing"])
fg = seg_features(clf, keys, gold, ex["finishing"], ex["previous"])
fp = seg_features(clf, keys, pred, ex["finishing"], ex["previous"])
if fg is not None and fp is not None:
mistakes += 1
scale = lr * (0.5 if ex["kind"].endswith("/prefix") else 1.0)
+27 -2
View File
@@ -6,14 +6,16 @@ tests); this file only translates PIME key events and replies.
"""
import json
import os
import time
from keycodes import * # noqa: F401,F403 (VK_* constants, provided by PIME)
from textService import TextService
from .core.classifier import Classifier, LearnedPreferences
from .core.engine import BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, TAB, UP, CompositionEngine
from .core.engine import BACKSPACE, DOWN, ENTER, ESCAPE, LEFT, RIGHT, SHIFT_TAP, TAB, UP, CompositionEngine
from .core.english import EnglishModel
from .core.lexicon import Lexicon
from .core.shift_tap import ShiftTapDetector
_HERE = os.path.dirname(os.path.abspath(__file__))
_DATA = os.path.join(_HERE, "data")
@@ -67,6 +69,7 @@ class AutoZhuyinTextService(TextService):
def __init__(self, client):
TextService.__init__(self, client)
self.engine = CompositionEngine(_shared_classifier())
self.shift_tap = ShiftTapDetector()
# --- lifecycle ---
@@ -105,6 +108,13 @@ class AutoZhuyinTextService(TextService):
return bool(self.engine.keys or self.engine.candidates)
def filterKeyDown(self, ev):
if ev.keyCode == VK_SHIFT: # noqa: F405
if ev.isKeyDown(VK_CONTROL) or ev.isKeyDown(VK_MENU): # noqa: F405
self.shift_tap.key_pressed()
else:
self.shift_tap.shift_down(time.monotonic())
return False
self.shift_tap.key_pressed()
# Shortcuts (Ctrl/Alt) go to the app.
if ev.isKeyDown(VK_CONTROL) or ev.isKeyDown(VK_MENU): # noqa: F405
return False
@@ -121,6 +131,21 @@ class AutoZhuyinTextService(TextService):
self._update(commit)
return handled
def filterKeyUp(self, ev):
# Shift pressed and released alone: next word is English (or flip the current one).
if ev.keyCode == VK_SHIFT: # noqa: F405
return self.shift_tap.shift_up(time.monotonic())
return False
def onKeyUp(self, ev):
if ev.keyCode != VK_SHIFT: # noqa: F405
return False
self.engine.handle(SHIFT_TAP)
self._update(None)
if not self.engine.keys:
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
return True
# --- replies ---
def _update(self, commit):
@@ -128,7 +153,7 @@ class AutoZhuyinTextService(TextService):
self.setCommitString(commit)
marked = self.engine.marked_text
self.setCompositionString(marked)
self.setCompositionCursor(len(marked))
self.setCompositionCursor(self.engine.cursor_offset)
if self.engine.showing_candidates:
page, cursor = self.engine.page()
self.setCandidateList(page)
+6 -10
View File
@@ -30,7 +30,6 @@ class LearnedPreferences:
class Classifier:
context_weight = 1.5
def __init__(self, lexicon, english, learned=None, weights=None):
self.lexicon = lexicon
@@ -49,20 +48,17 @@ class Classifier:
except (OSError, ValueError):
return {}
def segment_features(self, seg, is_last, after_other, finishing):
"""(zhuyin features or None, literal features or None) for one segment."""
def segment_features(self, seg, is_last, after_other, finishing, previous=None):
"""(zhuyin features or None, literal features or None) for one segment.
`previous` (mode of the text committed just before) only affects the first segment."""
seg = list(seg)
return (zhuyin_features(seg, self.lexicon, is_last, after_other, finishing),
literal_features(seg, self.english, is_last, after_other, finishing))
return (zhuyin_features(seg, self.lexicon, is_last, after_other, finishing, previous),
literal_features(seg, self.english, is_last, after_other, finishing, previous))
def _scores(self, seg, is_last, after_other, finishing, previous, whole, weights):
fz, fl = self.segment_features(seg, is_last, after_other, finishing)
fz, fl = self.segment_features(seg, is_last, after_other, finishing, previous)
zh = dot(weights, fz) if fz is not None else NEG_INF
lit = dot(weights, fl) if fl is not None else NEG_INF
if previous == ZHUYIN:
zh += self.context_weight
elif previous == LITERAL:
lit += self.context_weight
if whole:
b = self.learned.bias("".join(seg))
if b > 0:
+99 -23
View File
@@ -12,7 +12,8 @@ from .lexicon import Converter
BACKSPACE, ENTER, ESCAPE, TAB = "<backspace>", "<enter>", "<escape>", "<tab>"
UP, DOWN, LEFT, RIGHT = "<up>", "<down>", "<left>", "<right>"
SPECIAL_KEYS = {BACKSPACE, ENTER, ESCAPE, TAB, UP, DOWN, LEFT, RIGHT}
SHIFT_TAP = "<shift-tap>" # Shift pressed and released on its own
SPECIAL_KEYS = {BACKSPACE, ENTER, ESCAPE, TAB, UP, DOWN, LEFT, RIGHT, SHIFT_TAP}
FULLWIDTH = {
"<": ",", ">": "。", "?": "?", "!": "!", ":": ":", '"': "「", "(": "(", ")": ")",
@@ -33,10 +34,15 @@ class CompositionEngine:
self.now = now
self.keys = []
self.forced_mode = None
# English mode, toggled by a Shift tap when nothing is being typed: keys go straight
# to the app (no composition, no underline) until Shift is tapped again.
self.english_mode = False
self.fixed = {} # zhuyin segment start -> {(syl_start, syl_end): text}
self.candidates = []
self.candidate_index = 0
self._candidate_segment_start = 0
# Composition cursor for choosing candidates: index into _cursor_stops(), None = end.
self.cursor = None
self._previous_mode = None
self._previous_time = float("-inf")
self._cache_key = None
@@ -77,19 +83,61 @@ class CompositionEngine:
def page_start(self):
return (self.candidate_index // self.page_size) * self.page_size
def _convertible(self, seg, final):
"""(syllables converted to Chinese, pending Bopomofo) for a Zhuyin segment."""
start, end, _ = seg
p = bopomofo.parse(self.keys[start:end])
if final:
return [s.syllable for s in p.syllables], ""
return [s.syllable for s in p.completed], (p.pending.syllable.reading if p.pending else "")
def _conversion(self, seg, final):
syl, pending = self._convertible(seg, final)
fx = {k: v for k, v in self.fixed.get(seg[0], {}).items() if k[1] <= len(syl)}
return self.converter.convert(syl, fx), pending
def _render(self, seg, final):
start, end, mode = seg
seg_keys = self.keys[start:end]
if mode == LITERAL:
return "".join(seg_keys)
p = bopomofo.parse(seg_keys)
fixed_here = self.fixed.get(start, {})
syl = [s.syllable for s in (p.syllables if final else p.completed)]
fx = {k: v for k, v in fixed_here.items() if k[1] <= len(syl)}
text = "".join(t for _, t in self.converter.convert(syl, fx))
if not final and p.pending:
text += p.pending.syllable.reading
return text
return "".join(self.keys[start:end])
converted, pending = self._conversion(seg, final)
return "".join(t for _, t in converted) + pending
# --- cursor ---
def _cursor_stops(self):
"""[(segment index, syllables before the stop)] for every Zhuyin syllable."""
segs = self.segments
out = []
for i, seg in enumerate(segs):
if seg[2] == ZHUYIN:
n = len(self._convertible(seg, i < len(segs) - 1)[0])
out.extend((i, k) for k in range(1, n + 1))
return out
@property
def cursor_offset(self):
"""Cursor position in marked_text (characters)."""
pieces = self.marked_pieces
stops = self._cursor_stops()
if self.cursor is None or not 0 <= self.cursor < len(stops):
return sum(len(t) for t, _ in pieces)
seg_index, local = stops[self.cursor]
offset = sum(len(t) for t, _ in pieces[:seg_index])
segs = self.segments
for (a, b), text in self._conversion(segs[seg_index], seg_index < len(segs) - 1)[0]:
if b <= local:
offset += len(text)
elif a < local:
offset += min(local - a, len(text))
return offset
def _move_cursor(self, delta):
stops = self._cursor_stops()
if not stops:
return
nxt = (len(stops) - 1 if self.cursor is None else self.cursor) + delta
self.cursor = None if nxt >= len(stops) - 1 else max(0, nxt)
@property
def marked_pieces(self):
@@ -102,10 +150,16 @@ class CompositionEngine:
# --- key handling ---
def _flip(self):
"""Forces the whole buffer into the other mode (a correction that is learned)."""
self.forced_mode = LITERAL if self.mode == ZHUYIN else ZHUYIN
def wants(self, key):
"""Whether handle(key) would consume the key, without changing state."""
if self.candidates:
if self.candidates or key == SHIFT_TAP:
return True
if self.english_mode and not self.keys:
return False # English mode: everything goes to the app as typed
if key in SPECIAL_KEYS:
return bool(self.keys)
if self.keys:
@@ -122,6 +176,7 @@ class CompositionEngine:
if key == BACKSPACE:
if not self.keys:
return PASS
self.cursor = None
self.keys.pop()
self.fixed = {k: v for k, v in self.fixed.items() if k < len(self.keys)}
if not self.keys:
@@ -129,7 +184,9 @@ class CompositionEngine:
self.fixed = {}
return CONSUMED
if key == ENTER:
return (True, self._commit()) if self.keys else PASS
if not self.keys:
return PASS
return True, self._commit()
if key == ESCAPE:
if not self.keys:
return PASS
@@ -138,14 +195,28 @@ class CompositionEngine:
if key == TAB:
if not self.keys:
return PASS
self.forced_mode = LITERAL if self.mode == ZHUYIN else ZHUYIN
self._flip()
return CONSUMED
if key == SHIFT_TAP:
if not self.keys:
# Toggle English mode; it stays on until the next Shift tap.
self.english_mode = not self.english_mode
self._previous_mode = LITERAL if self.english_mode else None
self._previous_time = self.now()
else:
self._flip()
return CONSUMED
if key == DOWN:
if not self.keys:
return PASS
self._open_candidates()
return CONSUMED
if key in (UP, LEFT, RIGHT):
if key in (LEFT, RIGHT):
if not self.keys:
return PASS
self._move_cursor(-1 if key == LEFT else 1)
return CONSUMED
if key == UP:
return CONSUMED if self.keys else PASS
return self._handle_character(key)
@@ -156,6 +227,9 @@ class CompositionEngine:
return bopomofo.parse(self.keys[start:end])
def _handle_character(self, c):
self.cursor = None # typing always continues at the end
if self.english_mode and not self.keys:
return PASS # English mode: every key goes to the app as typed
if c == " ":
if not self.keys:
return PASS
@@ -188,15 +262,16 @@ class CompositionEngine:
# --- candidates (for the last Zhuyin segment) ---
def _open_candidates(self):
"""Candidates for the phrase ending at the cursor (default: the last syllable)."""
stops = self._cursor_stops()
if not stops:
return
seg_index, local = stops[min(len(stops) - 1 if self.cursor is None else self.cursor, len(stops) - 1)]
segs = self.segments
if not segs or segs[-1][2] != ZHUYIN:
return
start, end, _ = segs[-1]
syl = [s.syllable for s in bopomofo.parse(self.keys[start:end]).completed]
if not syl:
return
self.candidates = self.converter.candidates(syl, len(syl))
self._candidate_segment_start = start
seg = segs[seg_index]
syl = self._convertible(seg, seg_index < len(segs) - 1)[0]
self.candidates = self.converter.candidates(syl, local)
self._candidate_segment_start = seg[0]
self.candidate_index = 0
def page(self):
@@ -244,6 +319,7 @@ class CompositionEngine:
def reset(self):
self.keys = []
self.cursor = None
self.forced_mode = None
self.fixed = {}
self.candidates = []
+9 -2
View File
@@ -41,6 +41,9 @@ DEFAULT_WEIGHTS = {
"lit.others": -1.5,
# Segmentation
"seg.switch": -3.0,
# Context: the first segment has the same mode as the previously committed text
"ctx.zh": 1.5,
"ctx.lit": 1.5,
}
@@ -52,7 +55,7 @@ def _add(f, k, v=1.0):
f[k] = f.get(k, 0.0) + v
def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=False):
def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=False, previous=None):
"""Features of `keys` read as Zhuyin, or None if impossible."""
parse = bopomofo.parse(keys)
if not is_last and parse.pending is not None:
@@ -99,10 +102,12 @@ def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=Fa
_add(f, "zh.unconfirmed")
if after_other:
_add(f, "seg.switch")
elif previous == "zhuyin":
_add(f, "ctx.zh")
return f
def literal_features(keys, english, is_last=True, after_other=False, finishing=False):
def literal_features(keys, english, is_last=True, after_other=False, finishing=False, previous=None):
"""Features of `keys` output as typed, or None if impossible."""
if not keys or " " in keys:
return None
@@ -110,6 +115,8 @@ def literal_features(keys, english, is_last=True, after_other=False, finishing=F
f = {"lit.bias": 1.0}
if after_other:
_add(f, "seg.switch")
elif previous == "literal":
_add(f, "ctx.lit")
if any(c.isupper() for c in keys):
_add(f, "lit.upper")
letters = "".join(c for c in keys if c.isalpha())
+24
View File
@@ -0,0 +1,24 @@
"""Detects Shift pressed and released on its own. Port of ShiftTapDetector.swift."""
class ShiftTapDetector:
max_duration = 0.5
def __init__(self):
self._pressed_at = None
self._used = False
def shift_down(self, time):
if self._pressed_at is None: # ignore auto-repeat
self._pressed_at = time
self._used = False
def key_pressed(self):
"""Any other key while Shift is held: Shift was used as a modifier."""
if self._pressed_at is not None:
self._used = True
def shift_up(self, time):
"""Returns True when this release completes a Shift tap."""
start, self._pressed_at = self._pressed_at, None
return start is not None and not self._used and time - start <= self.max_duration
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "自動注音 AutoZhuyin",
"version": "0.3.0",
"version": "0.6.1",
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
"locale": "zh-Hant-TW",
"fallbackLocale": "zh-TW",
+6 -2
View File
@@ -26,9 +26,13 @@
空白 注音:一聲;音節打完時送出中文。英文:送出單字加空白
Enter 送出目前組字
Tab 判斷錯了?按 Tab 在中/英之間翻轉,輸入法會記住這次修正
↓ 開啟候選字,按數字鍵 1–9 選字
← → 在組字區移動游標,選要改哪個字
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字
Esc 取消組字
Shift+字母 強制英文
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線),
再單按一次回到中英自動判斷,會出現「英文」/「中英自動」小提示;
打到一半時按,同 Tab 整段翻轉
Shift+字母 打大寫字母(也會強制英文)
< > ? ! : 中文語境下自動變成全形 ,。?!:
注音符號的順序打錯也沒關係,例如 ㄧㄒㄠˇ 會自動變成 ㄒㄧㄠˇ(小)。
+89 -2
View File
@@ -11,7 +11,8 @@ sys.path.insert(0, os.path.dirname(HERE))
from autozhuyin.core import bopomofo # noqa: E402
from autozhuyin.core.classifier import LITERAL, ZHUYIN, Classifier, LearnedPreferences # noqa: E402
from autozhuyin.core.engine import DOWN, ENTER, TAB, CompositionEngine # noqa: E402
from autozhuyin.core.engine import DOWN, ENTER, LEFT, RIGHT, SHIFT_TAP, TAB, BACKSPACE, CompositionEngine # noqa: E402
from autozhuyin.core.shift_tap import ShiftTapDetector # noqa: E402
from autozhuyin.core.english import EnglishModel # noqa: E402
from autozhuyin.core.lexicon import Converter, Lexicon # noqa: E402
@@ -41,7 +42,7 @@ def make_engine():
def type_keys(s, e):
out = ""
for c in s:
key = ENTER if c == "\n" else TAB if c == "\t" else c
key = {"\n": ENTER, "\t": TAB, "⇧": SHIFT_TAP, "←": LEFT, "→": RIGHT, "↓": DOWN}.get(c, c)
handled, commit = e.handle(key)
if commit is not None:
out += commit
@@ -192,5 +193,91 @@ class MixedSegmentTests(unittest.TestCase):
self.assertLess(per_key, 0.05)
class ShiftTapTests(unittest.TestCase):
def test_toggles_english_mode(self):
e = make_engine()
self.assertEqual(type_keys("⇧ru su ", e), "ru su ")
self.assertTrue(e.english_mode)
self.assertEqual(e.marked_text, "")
self.assertEqual(e.handle(ENTER), (False, None))
self.assertTrue(e.english_mode)
self.assertEqual(type_keys("⇧su3cl3\n", e), "你好")
self.assertFalse(e.english_mode)
def test_tap_twice_cancels(self):
self.assertEqual(type_keys("⇧⇧su3cl3\n", make_engine()), "你好")
def test_mid_word_flips_and_learns(self):
e = make_engine()
type_keys("so", e)
e.handle(SHIFT_TAP)
self.assertEqual(e.mode, ZHUYIN)
self.assertFalse(e.english_mode)
type_keys("\n", e)
self.assertEqual(e.classifier.learned.weights["so"], 1)
def test_english_mode_not_learned(self):
e = make_engine()
type_keys("⇧hello ", e)
self.assertEqual(e.classifier.learned.weights, {})
def test_backspace_goes_to_app(self):
e = make_engine()
e.handle(SHIFT_TAP)
self.assertEqual(e.handle(BACKSPACE), (False, None))
self.assertTrue(e.english_mode)
def test_detector(self):
d = ShiftTapDetector()
d.shift_down(0)
self.assertTrue(d.shift_up(0.1))
d.shift_down(1)
d.key_pressed()
self.assertFalse(d.shift_up(1.1))
d.shift_down(2)
self.assertFalse(d.shift_up(3))
class CursorTests(unittest.TestCase):
def test_moves_and_clamps(self):
e = make_engine()
type_keys("rup wu0 wu0 fu4", e)
self.assertEqual(e.cursor_offset, 4)
type_keys("←", e)
self.assertEqual(e.cursor_offset, 3)
type_keys("←←←←", e)
self.assertEqual(e.cursor_offset, 1)
type_keys("→→→→", e)
self.assertEqual(e.cursor_offset, 4)
def test_choose_candidate_in_the_middle(self):
e = make_engine()
type_keys("rup wu0 wu0 fu4←←↓", e)
self.assertTrue(e.showing_candidates)
self.assertTrue(all(r[1] == 2 for r, _ in e.candidates))
e.select_candidate([t for _, t in e.candidates].index("添"))
self.assertEqual(type_keys("\n", e), "今添天氣")
def test_typing_returns_cursor_to_end(self):
e = make_engine()
type_keys("rup wu0 ←", e)
self.assertEqual(e.cursor_offset, 1)
type_keys("wu0 ", e)
self.assertEqual(e.cursor_offset, len(e.marked_text))
def test_candidates_for_chinese_before_english(self):
e = make_engine()
type_keys("su3cl3hello↓", e)
self.assertTrue(e.showing_candidates)
class FirstToneAfterEnglishTests(unittest.TestCase):
def test_first_tone_after_english(self):
for keys in ["ru", "t", "gj", "vup", "tj", "el"]:
e = make_engine()
self.assertEqual(type_keys("hello " + keys + " ", e), "hello ", keys)
self.assertEqual(len(e.marked_text), 1, keys)
if __name__ == "__main__":
unittest.main()
+34 -2
View File
@@ -36,6 +36,8 @@ class FakeTextService:
def setCandidateList(self, c): self.reply["candidateList"] = c
def setCandidateCursor(self, p): self.reply["candidateCursor"] = p
def showMessage(self, message, duration=3): self.reply["showMessage"] = message
def setShowCandidates(self, show):
self.showCandidates = show
self.reply["showCandidates"] = show
@@ -51,8 +53,8 @@ VK = {"\n": 0x0D, "\t": 0x09, " ": 0x20, "\b": 0x08, "↓": 0x28}
class FakeKeyEvent:
def __init__(self, ch, shift=False, ctrl=False):
self.keyCode = VK.get(ch, ord(ch.upper()) if ch.isalnum() else 0xBA)
self.charCode = 0 if ch in ("\n", "\t", "\b", "↓") else ord(ch)
self.keyCode = 0x10 if ch == "⇧" else VK.get(ch, ord(ch.upper()) if ch.isalnum() else 0xBA)
self.charCode = 0 if ch in ("\n", "\t", "\b", "↓", "⇧") else ord(ch)
self.keyStates = [0] * 256
if shift:
self.keyStates[0x10] = 0x80
@@ -74,6 +76,17 @@ def press(svc, ch, **kw):
return True, svc.reply
def tap_shift(svc):
"""Shift down then up with nothing in between, as PIME delivers it."""
svc.reply = {}
down = FakeKeyEvent("⇧", shift=True)
assert not svc.filterKeyDown(down)
up = FakeKeyEvent("⇧")
if svc.filterKeyUp(up):
svc.onKeyUp(up)
return svc.reply
def type_text(svc, text):
out = ""
for ch in text:
@@ -123,5 +136,24 @@ class WrapperTests(unittest.TestCase):
self.assertEqual(out + reply.get("commitString", ""), "Hi ")
class ShiftTapWrapperTests(unittest.TestCase):
def setUp(self):
self.svc = AutoZhuyinTextService(client=None)
self.svc.onActivate()
def test_tap_forces_english_lowercase(self):
reply = tap_shift(self.svc)
self.assertEqual(reply.get("showMessage"), "英文")
self.assertEqual(type_text(self.svc, "ru su "), "ru su ")
self.assertEqual(tap_shift(self.svc).get("showMessage"), "中英自動")
def test_shift_with_letter_is_not_a_tap(self):
self.svc.filterKeyDown(FakeKeyEvent("⇧", shift=True))
press(self.svc, "H", shift=True)
up = FakeKeyEvent("⇧")
self.assertFalse(self.svc.filterKeyUp(up))
self.assertFalse(self.svc.engine.english_mode)
if __name__ == "__main__":
unittest.main()