Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1dacb8f164 | ||
|
|
bfeed22916 | ||
|
|
bb9cb0c4e0 | ||
|
|
2c5c70c066 | ||
|
|
17f7d6f50d | ||
|
|
7e173a1da8 |
@@ -12,7 +12,7 @@
|
|||||||
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
|
- **中英自動判斷**:每按一個鍵就重新判斷,組字區即時顯示結果(Mac 版英文以藍字顯示)
|
||||||
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
|
- **中英混打不用空白**:`ji3hello` → 我hello、`hellosu3cl3` → hello你好,同一段內會自動切分
|
||||||
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
|
- **判斷錯了按 Tab**:在中/英之間翻轉,輸入法會記住這組按鍵,下次自動判斷對
|
||||||
- **單按 Shift 強制英文**:要打英文時先單按一下 Shift,接下來這個字照打、保持小寫,而且直接送出、不顯示底線
|
- **單按 Shift 切換英文模式**:英文模式下按鍵直接送出、保持小寫、不顯示底線,再單按一次 Shift 回到中英自動判斷
|
||||||
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
|
- **注音順序打錯也能修正**:`ㄧㄒㄠˇ`、`ㄒㄠㄧˇ` 都會變成 `ㄒㄧㄠˇ`(小)
|
||||||
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
|
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
|
||||||
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
|
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
|
||||||
@@ -54,7 +54,7 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
|||||||
| ← → | 在組字區移動游標,選要改哪個字 |
|
| ← → | 在組字區移動游標,選要改哪個字 |
|
||||||
| ↓ | 開啟游標前那個字或詞的候選字(游標預設在最後),數字鍵 1–9 選字,Esc 關閉 |
|
| ↓ | 開啟游標前那個字或詞的候選字(游標預設在最後),數字鍵 1–9 選字,Esc 關閉 |
|
||||||
| Esc | 取消組字 |
|
| Esc | 取消組字 |
|
||||||
| **單按 Shift** | 沒在打字時:下一個字強制英文(保持小寫),字母直接送出、沒有底線;按空白、Enter 或其他符號後恢復自動判斷,再按一次 Shift 取消。打到一半時:同 Tab,整段翻轉 |
|
| **單按 Shift** | 沒在打字時:切換**英文模式**,按鍵直接送出、保持小寫、沒有底線,一直維持到再單按一次 Shift 才回到中英自動判斷。打到一半時:同 Tab,整段翻轉 |
|
||||||
| Shift + 字母 | 打大寫字母(也會強制英文) |
|
| Shift + 字母 | 打大寫字母(也會強制英文) |
|
||||||
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
|
| `< > ? ! : ( ) [ ]` | 中文語境下自動轉成全形 `,。?!:()「」` |
|
||||||
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
|
| `, . / ; -` | 英文單字後是標點;在注音中是 ㄝ ㄡ ㄥ ㄤ ㄦ(例如 `2;3` = 檔) |
|
||||||
@@ -144,12 +144,12 @@ python3 training/make_parity.py # 權重或特徵改了之後都要重
|
|||||||
|
|
||||||
目前的結果(10,000 筆測試資料):
|
目前的結果(10,000 筆測試資料):
|
||||||
|
|
||||||
| | 送出時整段判斷正確 | 打字過程中顯示正確 |
|
| | 送出時整段判斷正確 | 打字過程中顯示正確 | 跟前一個字語言不同時 |
|
||||||
|---|---|---|
|
|---|---|---|---|
|
||||||
| 手調權重 | 97.58% | 86.15% |
|
| 手調權重 | 97.84% | 88.19% | 96.2% |
|
||||||
| 訓練後 | **99.62%** | 87.81% |
|
| 訓練後 | **99.52%** | **91.49%** | **99.3%** |
|
||||||
|
|
||||||
送出時的錯誤率從 2.4% 降到 0.4%。「打字過程中」本來就有歧義(例如 `ji3h` 的 `h` 可能是英文的開頭,也可能是注音 ㄘ),所以準確率比較低,但最後送出的結果才是重點。
|
訓練資料也模擬了「前一個字是中文或英文」和「按下空白(一聲)的那一刻」,所以「前一個字」的影響(`ctx.zh` / `ctx.lit`)也是訓練出來的,不是手調的。「打字過程中」本來就有歧義(例如 `ji3h` 的 `h` 可能是英文的開頭,也可能是注音 ㄘ),所以準確率比較低,但最後送出的結果才是重點。
|
||||||
|
|
||||||
新增特徵時,Swift 和 Python 兩邊都要加,預設權重設成 0,然後重新訓練。
|
新增特徵時,Swift 和 Python 兩邊都要加,預設權重設成 0,然後重新訓練。
|
||||||
|
|
||||||
|
|||||||
@@ -15,9 +15,9 @@
|
|||||||
<key>CFBundlePackageType</key>
|
<key>CFBundlePackageType</key>
|
||||||
<string>APPL</string>
|
<string>APPL</string>
|
||||||
<key>CFBundleShortVersionString</key>
|
<key>CFBundleShortVersionString</key>
|
||||||
<string>0.5.1</string>
|
<string>0.6.2</string>
|
||||||
<key>CFBundleVersion</key>
|
<key>CFBundleVersion</key>
|
||||||
<string>7</string>
|
<string>10</string>
|
||||||
<key>InputMethodConnectionName</key>
|
<key>InputMethodConnectionName</key>
|
||||||
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
||||||
<key>InputMethodServerControllerClass</key>
|
<key>InputMethodServerControllerClass</key>
|
||||||
|
|||||||
@@ -1,39 +1,41 @@
|
|||||||
{
|
{
|
||||||
"about": "AutoZhuyin classifier weights, learned by training/train.py",
|
"about": "AutoZhuyin classifier weights, learned by training/train.py",
|
||||||
"trained": "2026-11-11",
|
"trained": "2026-11-11",
|
||||||
"examples": 40001,
|
"examples": 40000,
|
||||||
"test_accuracy": {
|
"test_accuracy": {
|
||||||
"commit": 0.9962,
|
"commit": 0.9952,
|
||||||
"typing": 0.8781
|
"typing": 0.9149
|
||||||
},
|
},
|
||||||
"weights": {
|
"weights": {
|
||||||
"lit.bias": 0.4276,
|
"ctx.lit": 0.0916,
|
||||||
"lit.common": -0.3465,
|
"ctx.zh": 0.1349,
|
||||||
"lit.common_len": 0.81,
|
"lit.bias": 0.3982,
|
||||||
"lit.dict": -1.7902,
|
"lit.common": -0.2906,
|
||||||
"lit.dict_len": 0.6765,
|
"lit.common_len": 0.8046,
|
||||||
"lit.digits": -2.7403,
|
"lit.dict": -1.6662,
|
||||||
"lit.num_digits": -0.0751,
|
"lit.dict_len": 0.6441,
|
||||||
"lit.num_others": -0.7433,
|
"lit.digits": -2.9689,
|
||||||
"lit.others": -1.5685,
|
"lit.num_digits": -0.018,
|
||||||
"lit.prefix_common_len": 0.2082,
|
"lit.num_others": -0.5554,
|
||||||
"lit.prefix_dict_len": 0.0003,
|
"lit.others": -2.0002,
|
||||||
"lit.short": -0.2397,
|
"lit.prefix_common_len": 0.2831,
|
||||||
"lit.unknown_score": 0.213,
|
"lit.prefix_dict_len": 0.0299,
|
||||||
|
"lit.short": -0.2482,
|
||||||
|
"lit.unknown_score": 0.2479,
|
||||||
"lit.upper": 20.0,
|
"lit.upper": 20.0,
|
||||||
"seg.switch": -1.7203,
|
"seg.switch": -1.7596,
|
||||||
"zh.bias": 1.3522,
|
"zh.bias": 1.3423,
|
||||||
"zh.lexscore": 0.26,
|
"zh.lexscore": 0.2917,
|
||||||
"zh.pending_invalid": -5.0,
|
"zh.pending_invalid": -5.0,
|
||||||
"zh.pending_prefix": -0.9295,
|
"zh.pending_prefix": -0.9162,
|
||||||
"zh.phrase": 0.4595,
|
"zh.phrase": 0.4499,
|
||||||
"zh.reordered": -1.4348,
|
"zh.reordered": -1.3079,
|
||||||
"zh.stray": -5.0,
|
"zh.stray": -5.0,
|
||||||
"zh.toned_invalid": -5.0,
|
"zh.toned_invalid": -5.0,
|
||||||
"zh.toned_valid_lex": -0.9823,
|
"zh.toned_valid_lex": -0.9018,
|
||||||
"zh.toned_valid_nolex": -1.8742,
|
"zh.toned_valid_nolex": -1.6341,
|
||||||
"zh.unconfirmed": -0.7891,
|
"zh.unconfirmed": -0.7952,
|
||||||
"zh.untoned_mid_invalid": -5.0,
|
"zh.untoned_mid_invalid": -5.0,
|
||||||
"zh.untoned_mid_valid": -2.8009
|
"zh.untoned_mid_valid": -2.7479
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -11,22 +11,28 @@ enum Shared {
|
|||||||
|
|
||||||
static let classifier: Classifier = {
|
static let classifier: Classifier = {
|
||||||
let bundle = Bundle.main
|
let bundle = Bundle.main
|
||||||
|
let start = Date()
|
||||||
let lexicon = Lexicon()
|
let lexicon = Lexicon()
|
||||||
if let url = bundle.url(forResource: "lexicon", withExtension: "txt"),
|
if let url = bundle.url(forResource: "lexicon", withExtension: "txt"),
|
||||||
let text = try? String(contentsOf: url, encoding: .utf8) {
|
let data = try? Data(contentsOf: url) {
|
||||||
lexicon.load(text)
|
lexicon.load(data: data)
|
||||||
}
|
}
|
||||||
// Optional user phrases: ~/Library/Application Support/AutoZhuyin/user-phrases.txt
|
// Optional user phrases: ~/Library/Application Support/AutoZhuyin/user-phrases.txt
|
||||||
if let support = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first {
|
if let support = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first {
|
||||||
let userFile = support.appendingPathComponent("AutoZhuyin/user-phrases.txt")
|
let userFile = support.appendingPathComponent("AutoZhuyin/user-phrases.txt")
|
||||||
if let text = try? String(contentsOf: userFile, encoding: .utf8) { lexicon.load(text) }
|
if let text = try? String(contentsOf: userFile, encoding: .utf8) { lexicon.load(text) }
|
||||||
}
|
}
|
||||||
let english = EnglishModel(commonWordsURL: bundle.url(forResource: "english-common", withExtension: "txt"))
|
let english = EnglishModel(commonWordsURL: bundle.url(forResource: "english-common", withExtension: "txt"),
|
||||||
|
dictionaryURL: bundle.url(forResource: "english-dict", withExtension: "txt"),
|
||||||
|
bigramURL: bundle.url(forResource: "english-bigram", withExtension: "json"))
|
||||||
let saved = UserDefaults.standard.dictionary(forKey: learnedKey) as? [String: Int] ?? [:]
|
let saved = UserDefaults.standard.dictionary(forKey: learnedKey) as? [String: Int] ?? [:]
|
||||||
let learned = LearnedPreferences(weights: saved) { weights in
|
let learned = LearnedPreferences(weights: saved) { weights in
|
||||||
UserDefaults.standard.set(weights, forKey: learnedKey)
|
UserDefaults.standard.set(weights, forKey: learnedKey)
|
||||||
}
|
}
|
||||||
let trained = Features.loadWeights(from: bundle.url(forResource: "classifier-weights", withExtension: "json"))
|
let trained = Features.loadWeights(from: bundle.url(forResource: "classifier-weights", withExtension: "json"))
|
||||||
|
NSLog("AutoZhuyin: data loaded in %.0f ms", Date().timeIntervalSince(start) * 1000)
|
||||||
|
// Only needed for untoned syllables; build it off the startup path.
|
||||||
|
DispatchQueue.global(qos: .utility).async { lexicon.prepareTonelessIndex() }
|
||||||
return Classifier(lexicon: lexicon, english: english, learned: learned, weights: trained)
|
return Classifier(lexicon: lexicon, english: english, learned: learned, weights: trained)
|
||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -58,9 +58,6 @@ public final class Classifier {
|
|||||||
/// Feature weights: the hand-tuned defaults overlaid with trained ones.
|
/// Feature weights: the hand-tuned defaults overlaid with trained ones.
|
||||||
public private(set) var weights: FeatureVector
|
public private(set) var weights: FeatureVector
|
||||||
|
|
||||||
/// How much the previous segment's mode pulls the current one.
|
|
||||||
public var contextWeight = 1.5
|
|
||||||
|
|
||||||
public init(lexicon: Lexicon, english: EnglishModel, learned: LearnedPreferences = LearnedPreferences(),
|
public init(lexicon: Lexicon, english: EnglishModel, learned: LearnedPreferences = LearnedPreferences(),
|
||||||
weights: FeatureVector = [:]) {
|
weights: FeatureVector = [:]) {
|
||||||
self.lexicon = lexicon
|
self.lexicon = lexicon
|
||||||
@@ -69,23 +66,22 @@ public final class Classifier {
|
|||||||
self.weights = Features.defaultWeights.merging(weights) { _, new in new }
|
self.weights = Features.defaultWeights.merging(weights) { _, new in new }
|
||||||
}
|
}
|
||||||
|
|
||||||
/// (Zhuyin features or nil, literal features or nil) for one segment.
|
/// (Zhuyin features or nil, literal features or nil) for one segment. `previous` (mode of
|
||||||
public func segmentFeatures(_ seg: [Character], isLast: Bool, afterOther: Bool,
|
/// the text committed just before) only affects the first segment.
|
||||||
finishing: Bool) -> (FeatureVector?, FeatureVector?) {
|
public func segmentFeatures(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
|
||||||
(Features.zhuyin(seg, lexicon: lexicon, isLast: isLast, afterOther: afterOther, finishing: finishing),
|
previous: InputMode? = nil) -> (FeatureVector?, FeatureVector?) {
|
||||||
Features.literal(seg, english: english, isLast: isLast, afterOther: afterOther, finishing: finishing))
|
(Features.zhuyin(seg, lexicon: lexicon, isLast: isLast, afterOther: afterOther,
|
||||||
|
finishing: finishing, previous: previous),
|
||||||
|
Features.literal(seg, english: english, isLast: isLast, afterOther: afterOther,
|
||||||
|
finishing: finishing, previous: previous))
|
||||||
}
|
}
|
||||||
|
|
||||||
private func scores(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
|
private func scores(_ seg: [Character], isLast: Bool, afterOther: Bool, finishing: Bool,
|
||||||
previous: InputMode?, whole: Bool) -> [Double] {
|
previous: InputMode?, whole: Bool) -> [Double] {
|
||||||
let (fz, fl) = segmentFeatures(seg, isLast: isLast, afterOther: afterOther, finishing: finishing)
|
let (fz, fl) = segmentFeatures(seg, isLast: isLast, afterOther: afterOther,
|
||||||
|
finishing: finishing, previous: previous)
|
||||||
var zh = fz.map { Features.dot(weights, $0) } ?? -.infinity
|
var zh = fz.map { Features.dot(weights, $0) } ?? -.infinity
|
||||||
var lit = fl.map { Features.dot(weights, $0) } ?? -.infinity
|
var lit = fl.map { Features.dot(weights, $0) } ?? -.infinity
|
||||||
switch previous {
|
|
||||||
case .zhuyin?: zh += contextWeight
|
|
||||||
case .literal?: lit += contextWeight
|
|
||||||
case nil: break
|
|
||||||
}
|
|
||||||
if whole {
|
if whole {
|
||||||
let b = learned.bias(for: String(seg))
|
let b = learned.bias(for: String(seg))
|
||||||
if b > 0 { zh += b } else { lit -= b }
|
if b > 0 { zh += b } else { lit -= b }
|
||||||
|
|||||||
@@ -35,12 +35,12 @@ public final class CompositionEngine {
|
|||||||
public let converter: Converter
|
public let converter: Converter
|
||||||
|
|
||||||
public private(set) var keys: [Character] = []
|
public private(set) var keys: [Character] = []
|
||||||
/// Set by Tab / Shift tap: overrides the classifier for the whole buffer.
|
/// Set by Tab / a Shift tap while typing: overrides the classifier for the whole
|
||||||
|
/// buffer. It is a correction, learned on commit.
|
||||||
public private(set) var forcedMode: InputMode?
|
public private(set) var forcedMode: InputMode?
|
||||||
/// True when forcedMode corrects the classifier (learned on commit); false when the
|
/// English mode, toggled by a Shift tap when nothing is being typed. Keys go straight
|
||||||
/// user forced English before typing (Shift tap on an empty buffer), which is not a
|
/// to the app (no composition, no underline) until Shift is tapped again.
|
||||||
/// correction and is not learned.
|
public private(set) var isEnglishMode = false
|
||||||
private var forcedIsCorrection = false
|
|
||||||
/// User-picked phrases: Zhuyin segment start (key index) → syllable range → text.
|
/// User-picked phrases: Zhuyin segment start (key index) → syllable range → text.
|
||||||
private var fixed: [Int: [Range<Int>: String]] = [:]
|
private var fixed: [Int: [Range<Int>: String]] = [:]
|
||||||
public private(set) var candidates: [Candidate] = []
|
public private(set) var candidates: [Candidate] = []
|
||||||
@@ -165,28 +165,25 @@ public final class CompositionEngine {
|
|||||||
cursor = nil
|
cursor = nil
|
||||||
keys.removeLast()
|
keys.removeLast()
|
||||||
fixed = fixed.filter { $0.key < keys.count }
|
fixed = fixed.filter { $0.key < keys.count }
|
||||||
if keys.isEmpty {
|
if keys.isEmpty { forcedMode = nil; fixed = [:] }
|
||||||
// A Shift-tap "next word is English" stays armed; a correction is dropped.
|
|
||||||
if forcedIsCorrection { forcedMode = nil; forcedIsCorrection = false }
|
|
||||||
fixed = [:]
|
|
||||||
}
|
|
||||||
return .consumed
|
return .consumed
|
||||||
case .enter:
|
case .enter:
|
||||||
guard !keys.isEmpty else { endDirectEnglish(); return .passThrough }
|
guard !keys.isEmpty else { return .passThrough }
|
||||||
return KeyResult(handled: true, commit: commitCurrent())
|
return KeyResult(handled: true, commit: commitCurrent())
|
||||||
case .escape:
|
case .escape:
|
||||||
guard !keys.isEmpty else { endDirectEnglish(); return .passThrough }
|
guard !keys.isEmpty else { return .passThrough }
|
||||||
reset()
|
reset()
|
||||||
return .consumed
|
return .consumed
|
||||||
case .tab:
|
case .tab:
|
||||||
guard !keys.isEmpty else { endDirectEnglish(); return .passThrough }
|
guard !keys.isEmpty else { return .passThrough }
|
||||||
flip()
|
flip()
|
||||||
return .consumed
|
return .consumed
|
||||||
case .shiftTap:
|
case .shiftTap:
|
||||||
if keys.isEmpty {
|
if keys.isEmpty {
|
||||||
// Next word is English, typed as-is (lowercase stays lowercase). Tap again to cancel.
|
// Toggle English mode; it stays on until the next Shift tap.
|
||||||
forcedMode = forcedMode == .literal ? nil : .literal
|
isEnglishMode.toggle()
|
||||||
forcedIsCorrection = false
|
previousMode = isEnglishMode ? .literal : nil
|
||||||
|
previousCommitTime = now()
|
||||||
} else {
|
} else {
|
||||||
flip()
|
flip()
|
||||||
}
|
}
|
||||||
@@ -211,37 +208,12 @@ public final class CompositionEngine {
|
|||||||
/// Forces the whole buffer into the other mode (a correction that is learned).
|
/// Forces the whole buffer into the other mode (a correction that is learned).
|
||||||
private func flip() {
|
private func flip() {
|
||||||
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
|
forcedMode = (mode == .zhuyin) ? .literal : .zhuyin
|
||||||
forcedIsCorrection = true
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// True while the user has forced English for the next word (Shift tap). In this
|
|
||||||
/// mode keys go straight to the app with no composition (no underline).
|
|
||||||
public var isEnglishArmed: Bool { keys.isEmpty && forcedMode == .literal && !forcedIsCorrection }
|
|
||||||
|
|
||||||
/// Leaves forced-English mode once the word is done.
|
|
||||||
private func endDirectEnglish() {
|
|
||||||
guard isEnglishArmed else { return }
|
|
||||||
forcedMode = nil
|
|
||||||
previousMode = .literal
|
|
||||||
previousCommitTime = now()
|
|
||||||
}
|
|
||||||
|
|
||||||
private static let englishWordPunctuation: Set<Character> = ["'", ".", ",", "-", "/", ";"]
|
|
||||||
|
|
||||||
private func handleCharacter(_ c: Character) -> KeyResult {
|
private func handleCharacter(_ c: Character) -> KeyResult {
|
||||||
cursor = nil // typing always continues at the end
|
cursor = nil // typing always continues at the end
|
||||||
if isEnglishArmed {
|
// English mode: every key goes to the app as typed.
|
||||||
// Forced English: insert directly. Space ends the word; other symbols end it too.
|
if isEnglishMode && keys.isEmpty { return .passThrough }
|
||||||
if c == " " {
|
|
||||||
endDirectEnglish()
|
|
||||||
return KeyResult(handled: true, commit: " ")
|
|
||||||
}
|
|
||||||
if c.isASCII && (c.isLetter || c.isNumber || CompositionEngine.englishWordPunctuation.contains(c)) {
|
|
||||||
return KeyResult(handled: true, commit: String(c))
|
|
||||||
}
|
|
||||||
endDirectEnglish()
|
|
||||||
return .passThrough
|
|
||||||
}
|
|
||||||
if c == " " {
|
if c == " " {
|
||||||
guard !keys.isEmpty else { return .passThrough }
|
guard !keys.isEmpty else { return .passThrough }
|
||||||
let segs = segments(of: keys, finishing: true)
|
let segs = segments(of: keys, finishing: true)
|
||||||
@@ -338,7 +310,7 @@ public final class CompositionEngine {
|
|||||||
private func commitCurrent(_ given: [Segment]? = nil) -> String {
|
private func commitCurrent(_ given: [Segment]? = nil) -> String {
|
||||||
let segs = given ?? segments
|
let segs = given ?? segments
|
||||||
let text = segs.map { render($0, final: true) }.joined()
|
let text = segs.map { render($0, final: true) }.joined()
|
||||||
if let forced = forcedMode, forcedIsCorrection {
|
if let forced = forcedMode {
|
||||||
classifier.learned.record(String(keys), as: forced)
|
classifier.learned.record(String(keys), as: forced)
|
||||||
}
|
}
|
||||||
previousMode = segs.last?.mode
|
previousMode = segs.last?.mode
|
||||||
@@ -356,7 +328,6 @@ public final class CompositionEngine {
|
|||||||
keys = []
|
keys = []
|
||||||
cursor = nil
|
cursor = nil
|
||||||
forcedMode = nil
|
forcedMode = nil
|
||||||
forcedIsCorrection = false
|
|
||||||
fixed = [:]
|
fixed = [:]
|
||||||
candidates = []
|
candidates = []
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,33 +8,63 @@ import Foundation
|
|||||||
/// - a letter-bigram model trained on those words, for words not in either list.
|
/// - a letter-bigram model trained on those words, for words not in either list.
|
||||||
public final class EnglishModel {
|
public final class EnglishModel {
|
||||||
private var common = Set<String>()
|
private var common = Set<String>()
|
||||||
private var dictionary = Set<String>()
|
|
||||||
private var sortedWords: [String] = []
|
|
||||||
private var sortedCommon: [String] = []
|
private var sortedCommon: [String] = []
|
||||||
|
/// Dictionary words (lowercase, > 2 letters), sorted: membership and prefix checks are
|
||||||
|
/// binary searches, so no 230k-entry Set has to be built at startup.
|
||||||
|
private var sortedDict: [String] = []
|
||||||
private var bigram: [String: Double] = [:] // "ab" -> log2 P(b|a); "^a", "a$" for boundaries
|
private var bigram: [String: Double] = [:] // "ab" -> log2 P(b|a); "^a", "a$" for boundaries
|
||||||
private var unseenBigram = -12.0
|
private var unseenBigram = -12.0
|
||||||
|
|
||||||
public init(commonWords: [String], dictionaryWords: [String] = []) {
|
/// - Parameter bigram: precomputed table (english-bigram.json); trained from the words if nil.
|
||||||
|
public init(commonWords: [String], dictionaryWords: [String] = [],
|
||||||
|
bigram: (table: [String: Double], unseen: Double)? = nil) {
|
||||||
common = Set(commonWords.map { $0.lowercased() }.filter { !$0.isEmpty })
|
common = Set(commonWords.map { $0.lowercased() }.filter { !$0.isEmpty })
|
||||||
dictionary = Set(dictionaryWords.map { $0.lowercased() }.filter { $0.count > 2 })
|
|
||||||
sortedCommon = common.sorted()
|
sortedCommon = common.sorted()
|
||||||
sortedWords = common.union(dictionary).sorted()
|
var dict = dictionaryWords.map { $0.lowercased() }.filter { $0.count > 2 }
|
||||||
trainBigram()
|
if !zip(dict, dict.dropFirst()).allSatisfy({ $0 < $1 }) {
|
||||||
|
dict = Array(Set(dict)).sorted()
|
||||||
|
}
|
||||||
|
sortedDict = dict
|
||||||
|
if let bigram {
|
||||||
|
self.bigram = bigram.table
|
||||||
|
unseenBigram = bigram.unseen
|
||||||
|
} else {
|
||||||
|
trainBigram()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Loads common words from a bundle resource file and the system dictionary if present.
|
/// Loads the bundled data files. Falls back to the system word list and to training
|
||||||
public convenience init(commonWordsURL: URL?, systemDictionary: URL? = URL(fileURLWithPath: "/usr/share/dict/words")) {
|
/// the bigram table when the precomputed files are missing.
|
||||||
|
public convenience init(commonWordsURL: URL?, dictionaryURL: URL? = nil, bigramURL: URL? = nil) {
|
||||||
func lines(_ url: URL?) -> [String] {
|
func lines(_ url: URL?) -> [String] {
|
||||||
guard let url, let text = try? String(contentsOf: url, encoding: .utf8) else { return [] }
|
guard let url, let text = try? String(contentsOf: url, encoding: .utf8) else { return [] }
|
||||||
return text.split(whereSeparator: \.isNewline)
|
return text.split(whereSeparator: \.isNewline)
|
||||||
.map { $0.trimmingCharacters(in: .whitespaces) }
|
.map { $0.trimmingCharacters(in: .whitespaces) }
|
||||||
.filter { !$0.hasPrefix("#") }
|
.filter { !$0.isEmpty && !$0.hasPrefix("#") }
|
||||||
}
|
}
|
||||||
self.init(commonWords: lines(commonWordsURL), dictionaryWords: lines(systemDictionary))
|
var bigram: (table: [String: Double], unseen: Double)?
|
||||||
|
if let url = bigramURL, let data = try? Data(contentsOf: url),
|
||||||
|
let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
|
||||||
|
let table = obj["table"] as? [String: Double], let unseen = obj["unseen"] as? Double {
|
||||||
|
bigram = (table, unseen)
|
||||||
|
}
|
||||||
|
let dictURL = dictionaryURL.flatMap { FileManager.default.fileExists(atPath: $0.path) ? $0 : nil }
|
||||||
|
?? URL(fileURLWithPath: "/usr/share/dict/words")
|
||||||
|
self.init(commonWords: lines(commonWordsURL), dictionaryWords: lines(dictURL), bigram: bigram)
|
||||||
}
|
}
|
||||||
|
|
||||||
public func isCommon(_ w: String) -> Bool { common.contains(w.lowercased()) }
|
public func isCommon(_ w: String) -> Bool { common.contains(w.lowercased()) }
|
||||||
public func isWord(_ w: String) -> Bool { let l = w.lowercased(); return common.contains(l) || dictionary.contains(l) }
|
public func isWord(_ w: String) -> Bool { let l = w.lowercased(); return common.contains(l) || inDictionary(l) }
|
||||||
|
|
||||||
|
private func inDictionary(_ w: String) -> Bool {
|
||||||
|
let i = lowerBound(w, in: sortedDict)
|
||||||
|
return i < sortedDict.count && sortedDict[i] == w
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Some word strictly longer than `p` starts with it (common words or dictionary).
|
||||||
|
private func isPrefixOfWord(_ p: String) -> Bool {
|
||||||
|
hasPrefix(p, in: sortedCommon) || hasPrefix(p, in: sortedDict)
|
||||||
|
}
|
||||||
|
|
||||||
/// - Parameter complete: true when the user ended the word (space / punctuation),
|
/// - Parameter complete: true when the user ended the word (space / punctuation),
|
||||||
/// false while still typing (prefixes of words get partial credit).
|
/// false while still typing (prefixes of words get partial credit).
|
||||||
@@ -43,10 +73,10 @@ public final class EnglishModel {
|
|||||||
let n = Double(w.count)
|
let n = Double(w.count)
|
||||||
guard !w.isEmpty else { return 0 }
|
guard !w.isEmpty else { return 0 }
|
||||||
if common.contains(w) { return 3 + n }
|
if common.contains(w) { return 3 + n }
|
||||||
if dictionary.contains(w) { return 1 + 0.6 * n }
|
if inDictionary(w) { return 1 + 0.6 * n }
|
||||||
if !complete {
|
if !complete {
|
||||||
if hasPrefix(w, in: sortedCommon) { return 0.4 * n }
|
if hasPrefix(w, in: sortedCommon) { return 0.4 * n }
|
||||||
if hasPrefix(w, in: sortedWords) { return 0.2 * n }
|
if isPrefixOfWord(w) { return 0.2 * n }
|
||||||
}
|
}
|
||||||
// Unknown word: judge by how English-like its letter sequence is.
|
// Unknown word: judge by how English-like its letter sequence is.
|
||||||
let plaus = plausibility(w)
|
let plaus = plausibility(w)
|
||||||
@@ -63,12 +93,12 @@ public final class EnglishModel {
|
|||||||
if common.contains(w) {
|
if common.contains(w) {
|
||||||
f["lit.common"] = 1
|
f["lit.common"] = 1
|
||||||
f["lit.common_len"] = n
|
f["lit.common_len"] = n
|
||||||
} else if dictionary.contains(w) {
|
} else if inDictionary(w) {
|
||||||
f["lit.dict"] = 1
|
f["lit.dict"] = 1
|
||||||
f["lit.dict_len"] = n
|
f["lit.dict_len"] = n
|
||||||
} else if !complete && hasPrefix(w, in: sortedCommon) {
|
} else if !complete && hasPrefix(w, in: sortedCommon) {
|
||||||
f["lit.prefix_common_len"] = n
|
f["lit.prefix_common_len"] = n
|
||||||
} else if !complete && hasPrefix(w, in: sortedWords) {
|
} else if !complete && isPrefixOfWord(w) {
|
||||||
f["lit.prefix_dict_len"] = n
|
f["lit.prefix_dict_len"] = n
|
||||||
} else {
|
} else {
|
||||||
let plaus = plausibility(w)
|
let plaus = plausibility(w)
|
||||||
@@ -85,12 +115,17 @@ public final class EnglishModel {
|
|||||||
return total / Double(chars.count - 1)
|
return total / Double(chars.count - 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
private func hasPrefix(_ p: String, in words: [String]) -> Bool {
|
private func lowerBound(_ p: String, in words: [String]) -> Int {
|
||||||
var lo = 0, hi = words.count
|
var lo = 0, hi = words.count
|
||||||
while lo < hi {
|
while lo < hi {
|
||||||
let mid = (lo + hi) / 2
|
let mid = (lo + hi) / 2
|
||||||
if words[mid] < p { lo = mid + 1 } else { hi = mid }
|
if words[mid] < p { lo = mid + 1 } else { hi = mid }
|
||||||
}
|
}
|
||||||
|
return lo
|
||||||
|
}
|
||||||
|
|
||||||
|
private func hasPrefix(_ p: String, in words: [String]) -> Bool {
|
||||||
|
var lo = lowerBound(p, in: words)
|
||||||
// words[lo] is the first word >= p; skip an exact match, then test the next one.
|
// words[lo] is the first word >= p; skip an exact match, then test the next one.
|
||||||
if lo < words.count && words[lo] == p { lo += 1 }
|
if lo < words.count && words[lo] == p { lo += 1 }
|
||||||
return lo < words.count && words[lo].hasPrefix(p)
|
return lo < words.count && words[lo].hasPrefix(p)
|
||||||
@@ -99,7 +134,7 @@ public final class EnglishModel {
|
|||||||
private func trainBigram() {
|
private func trainBigram() {
|
||||||
var pair: [String: Double] = [:]
|
var pair: [String: Double] = [:]
|
||||||
var first: [String: Double] = [:]
|
var first: [String: Double] = [:]
|
||||||
for w in sortedWords where w.allSatisfy({ $0.isLetter && $0.isASCII }) {
|
for w in Set(sortedCommon + sortedDict).sorted() where w.allSatisfy({ $0.isLetter && $0.isASCII }) {
|
||||||
let weight = common.contains(w) ? 5.0 : 1.0
|
let weight = common.contains(w) ? 5.0 : 1.0
|
||||||
let chars = ["^"] + w.map(String.init) + ["$"]
|
let chars = ["^"] + w.map(String.init) + ["$"]
|
||||||
for i in 1..<chars.count {
|
for i in 1..<chars.count {
|
||||||
|
|||||||
@@ -42,6 +42,9 @@ public enum Features {
|
|||||||
"lit.others": -1.5,
|
"lit.others": -1.5,
|
||||||
// Segmentation
|
// Segmentation
|
||||||
"seg.switch": -3.0,
|
"seg.switch": -3.0,
|
||||||
|
// Context: the first segment has the same mode as the previously committed text
|
||||||
|
"ctx.zh": 1.5,
|
||||||
|
"ctx.lit": 1.5,
|
||||||
]
|
]
|
||||||
|
|
||||||
/// Reads {"weights": {...}} from a JSON file; empty if missing or invalid.
|
/// Reads {"weights": {...}} from a JSON file; empty if missing or invalid.
|
||||||
@@ -58,7 +61,8 @@ public enum Features {
|
|||||||
|
|
||||||
/// Features of `keys` read as Zhuyin, or nil if impossible.
|
/// Features of `keys` read as Zhuyin, or nil if impossible.
|
||||||
public static func zhuyin(_ keys: [Character], lexicon: Lexicon, isLast: Bool = true,
|
public static func zhuyin(_ keys: [Character], lexicon: Lexicon, isLast: Bool = true,
|
||||||
afterOther: Bool = false, finishing: Bool = false) -> FeatureVector? {
|
afterOther: Bool = false, finishing: Bool = false,
|
||||||
|
previous: InputMode? = nil) -> FeatureVector? {
|
||||||
let parse = ZhuyinParser.parse(keys)
|
let parse = ZhuyinParser.parse(keys)
|
||||||
// A Zhuyin segment followed by another must end with a tone.
|
// A Zhuyin segment followed by another must end with a tone.
|
||||||
if !isLast && parse.pending != nil { return nil }
|
if !isLast && parse.pending != nil { return nil }
|
||||||
@@ -107,18 +111,27 @@ public enum Features {
|
|||||||
if isLast && afterOther && !finishing && !parse.syllables.contains(where: \.toned) {
|
if isLast && afterOther && !finishing && !parse.syllables.contains(where: \.toned) {
|
||||||
add("zh.unconfirmed")
|
add("zh.unconfirmed")
|
||||||
}
|
}
|
||||||
if afterOther { add("seg.switch") }
|
if afterOther {
|
||||||
|
add("seg.switch")
|
||||||
|
} else if previous == .zhuyin {
|
||||||
|
add("ctx.zh")
|
||||||
|
}
|
||||||
return f
|
return f
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Features of `keys` output as typed, or nil if impossible.
|
/// Features of `keys` output as typed, or nil if impossible.
|
||||||
public static func literal(_ keys: [Character], english: EnglishModel, isLast: Bool = true,
|
public static func literal(_ keys: [Character], english: EnglishModel, isLast: Bool = true,
|
||||||
afterOther: Bool = false, finishing: Bool = false) -> FeatureVector? {
|
afterOther: Bool = false, finishing: Bool = false,
|
||||||
|
previous: InputMode? = nil) -> FeatureVector? {
|
||||||
if keys.isEmpty || keys.contains(" ") { return nil }
|
if keys.isEmpty || keys.contains(" ") { return nil }
|
||||||
let complete = !isLast || finishing
|
let complete = !isLast || finishing
|
||||||
var f: FeatureVector = ["lit.bias": 1]
|
var f: FeatureVector = ["lit.bias": 1]
|
||||||
func add(_ k: String, _ v: Double = 1) { f[k, default: 0] += v }
|
func add(_ k: String, _ v: Double = 1) { f[k, default: 0] += v }
|
||||||
if afterOther { add("seg.switch") }
|
if afterOther {
|
||||||
|
add("seg.switch")
|
||||||
|
} else if previous == .literal {
|
||||||
|
add("ctx.lit")
|
||||||
|
}
|
||||||
if keys.contains(where: { $0.isUppercase }) { add("lit.upper") }
|
if keys.contains(where: { $0.isUppercase }) { add("lit.upper") }
|
||||||
|
|
||||||
let letters = String(keys.filter { $0.isLetter })
|
let letters = String(keys.filter { $0.isLetter })
|
||||||
|
|||||||
@@ -11,58 +11,159 @@ public struct LexiconEntry: Equatable {
|
|||||||
/// File format, one entry per line (`#` starts a comment):
|
/// File format, one entry per line (`#` starts a comment):
|
||||||
/// ㄋㄧˇ-ㄏㄠˇ 你好 -4.2
|
/// ㄋㄧˇ-ㄏㄠˇ 你好 -4.2
|
||||||
/// The phrase/reading columns may be in either order; the score is optional.
|
/// The phrase/reading columns may be in either order; the score is optional.
|
||||||
|
///
|
||||||
|
/// Loading is on the input method's startup path, so the file is parsed as raw bytes,
|
||||||
|
/// and the tone-ignoring index (only needed for untoned syllables) is built lazily —
|
||||||
|
/// call `prepareTonelessIndex()` from a background queue to have it ready.
|
||||||
public final class Lexicon {
|
public final class Lexicon {
|
||||||
private var exact: [String: [LexiconEntry]] = [:]
|
private var exact: [String: [LexiconEntry]] = [:]
|
||||||
private var toneless: [String: [LexiconEntry]] = [:]
|
private var tonelessIndex: [String: [LexiconEntry]]?
|
||||||
|
private let lock = NSLock()
|
||||||
public private(set) var maxPhraseLength = 1
|
public private(set) var maxPhraseLength = 1
|
||||||
|
|
||||||
public init() {}
|
public init() {}
|
||||||
|
|
||||||
public convenience init(contentsOf url: URL) throws {
|
public convenience init(contentsOf url: URL) throws {
|
||||||
self.init()
|
self.init()
|
||||||
load(try String(contentsOf: url, encoding: .utf8))
|
load(data: try Data(contentsOf: url))
|
||||||
}
|
}
|
||||||
|
|
||||||
public func load(_ text: String) {
|
public func load(_ text: String) {
|
||||||
for line in text.split(whereSeparator: \.isNewline) {
|
load(data: Data(text.utf8))
|
||||||
let trimmed = line.trimmingCharacters(in: .whitespaces)
|
|
||||||
if trimmed.isEmpty || trimmed.hasPrefix("#") { continue }
|
|
||||||
let cols = trimmed.split(whereSeparator: { $0 == " " || $0 == "\t" }).map(String.init)
|
|
||||||
guard cols.count >= 2 else { continue }
|
|
||||||
let (reading, phrase) = Lexicon.isBopomofo(cols[0]) ? (cols[0], cols[1]) : (cols[1], cols[0])
|
|
||||||
guard Lexicon.isBopomofo(reading) else { continue }
|
|
||||||
let score = cols.count >= 3 ? Double(cols[2]) ?? -6 : -6
|
|
||||||
add(reading: reading, phrase: phrase, score: score)
|
|
||||||
}
|
|
||||||
for key in exact.keys { exact[key]?.sort { $0.score > $1.score } }
|
|
||||||
for key in toneless.keys { toneless[key]?.sort { $0.score > $1.score } }
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public func add(reading rawReading: String, phrase: String, score: Double) {
|
public func load(data: Data) {
|
||||||
let entry = LexiconEntry(phrase: phrase, score: score)
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
|
exact.reserveCapacity(exact.count + data.count / 30)
|
||||||
|
data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
|
||||||
|
let bytes = raw.bindMemory(to: UInt8.self)
|
||||||
|
let n = bytes.count
|
||||||
|
var i = 0
|
||||||
|
while i < n {
|
||||||
|
var end = i
|
||||||
|
while end < n && bytes[end] != 0x0A { end += 1 }
|
||||||
|
parseLine(bytes, i, end)
|
||||||
|
i = end + 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (key, entries) in exact where entries.count > 1 {
|
||||||
|
exact[key] = entries.sorted { $0.score > $1.score }
|
||||||
|
}
|
||||||
|
tonelessIndex = nil // rebuilt on demand
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Splits one line into up to three space/tab separated fields.
|
||||||
|
private func parseLine(_ b: UnsafeBufferPointer<UInt8>, _ start: Int, _ end: Int) {
|
||||||
|
var fields: [Range<Int>] = []
|
||||||
|
var i = start
|
||||||
|
while i < end && fields.count < 3 {
|
||||||
|
while i < end && (b[i] == 0x20 || b[i] == 0x09 || b[i] == 0x0D) { i += 1 }
|
||||||
|
if i >= end { break }
|
||||||
|
if fields.isEmpty && b[i] == 0x23 { return } // "#" comment
|
||||||
|
let s = i
|
||||||
|
while i < end && b[i] != 0x20 && b[i] != 0x09 && b[i] != 0x0D { i += 1 }
|
||||||
|
fields.append(s..<i)
|
||||||
|
}
|
||||||
|
guard fields.count >= 2 else { return }
|
||||||
|
func str(_ r: Range<Int>) -> String { String(decoding: UnsafeBufferPointer(rebasing: b[r]), as: UTF8.self) }
|
||||||
|
let (r, p) = Lexicon.isBopomofo(b, fields[0]) ? (fields[0], fields[1]) : (fields[1], fields[0])
|
||||||
|
guard Lexicon.isBopomofo(b, r) else { return }
|
||||||
|
let score = fields.count >= 3 ? Lexicon.parseDouble(b, fields[2]) ?? -6 : -6
|
||||||
|
// Neutral tone written as a prefix ("˙ㄉㄜ") needs normalising; detect it on bytes.
|
||||||
|
var prefixNeutral = b[r.lowerBound] == 0xCB
|
||||||
|
var k = r.lowerBound
|
||||||
|
while !prefixNeutral && k + 2 < r.upperBound {
|
||||||
|
if b[k] == 0x2D && b[k + 1] == 0xCB && b[k + 2] == 0x99 { prefixNeutral = true }
|
||||||
|
k += 1
|
||||||
|
}
|
||||||
|
addEntry(reading: str(r), phrase: str(p), score: score, normalise: prefixNeutral)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parses "-3.9151"-style decimals without creating a String.
|
||||||
|
private static func parseDouble(_ b: UnsafeBufferPointer<UInt8>, _ r: Range<Int>) -> Double? {
|
||||||
|
var i = r.lowerBound
|
||||||
|
var sign = 1.0
|
||||||
|
if i < r.upperBound && (b[i] == 0x2D || b[i] == 0x2B) { sign = b[i] == 0x2D ? -1 : 1; i += 1 }
|
||||||
|
var value = 0.0, scale = 1.0, digits = 0, fraction = false
|
||||||
|
while i < r.upperBound {
|
||||||
|
let c = b[i]
|
||||||
|
if c >= 0x30 && c <= 0x39 {
|
||||||
|
if fraction { scale /= 10; value += Double(c - 0x30) * scale } else { value = value * 10 + Double(c - 0x30) }
|
||||||
|
digits += 1
|
||||||
|
} else if c == 0x2E && !fraction {
|
||||||
|
fraction = true
|
||||||
|
} else {
|
||||||
|
return Double(String(decoding: UnsafeBufferPointer(rebasing: b[r]), as: UTF8.self))
|
||||||
|
}
|
||||||
|
i += 1
|
||||||
|
}
|
||||||
|
return digits > 0 ? sign * value : nil
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Adds one entry (call `load` for bulk data; this does not re-sort).
|
||||||
|
public func add(reading: String, phrase: String, score: Double) {
|
||||||
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
|
let key = addEntry(reading: reading, phrase: phrase, score: score, normalise: reading.contains("˙"))
|
||||||
|
exact[key]?.sort { $0.score > $1.score }
|
||||||
|
tonelessIndex = nil
|
||||||
|
}
|
||||||
|
|
||||||
|
@discardableResult
|
||||||
|
private func addEntry(reading rawReading: String, phrase: String, score: Double, normalise: Bool) -> String {
|
||||||
|
var reading = rawReading
|
||||||
// Some data sets write the neutral tone as a prefix ("˙ㄉㄜ"); we use a suffix.
|
// Some data sets write the neutral tone as a prefix ("˙ㄉㄜ"); we use a suffix.
|
||||||
let syllables = rawReading.split(separator: "-").map { s -> String in
|
if normalise {
|
||||||
s.hasPrefix("˙") ? String(s.dropFirst()) + "˙" : String(s)
|
reading = reading.split(separator: "-").map { s -> String in
|
||||||
|
s.hasPrefix("˙") ? String(s.dropFirst()) + "˙" : String(s)
|
||||||
|
}.joined(separator: "-")
|
||||||
}
|
}
|
||||||
let reading = syllables.joined(separator: "-")
|
maxPhraseLength = max(maxPhraseLength, reading.utf8.reduce(1) { $0 + ($1 == 0x2D ? 1 : 0) })
|
||||||
maxPhraseLength = max(maxPhraseLength, syllables.count)
|
if exact[reading] == nil {
|
||||||
if !(exact[reading]?.contains { $0.phrase == phrase } ?? false) {
|
exact[reading] = [LexiconEntry(phrase: phrase, score: score)]
|
||||||
exact[reading, default: []].append(entry)
|
} else if !exact[reading]!.contains(where: { $0.phrase == phrase }) {
|
||||||
|
exact[reading]!.append(LexiconEntry(phrase: phrase, score: score))
|
||||||
}
|
}
|
||||||
let key = Lexicon.stripTones(reading)
|
return reading
|
||||||
if let i = toneless[key]?.firstIndex(where: { $0.phrase == phrase }) {
|
}
|
||||||
toneless[key]![i].score = max(toneless[key]![i].score, score)
|
|
||||||
} else {
|
/// Builds the index that ignores tones (for untoned syllables). Safe to call from a
|
||||||
toneless[key, default: []].append(entry)
|
/// background queue; lookups build it themselves if it is not ready yet.
|
||||||
|
public func prepareTonelessIndex() {
|
||||||
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
|
_ = buildTonelessIfNeeded()
|
||||||
|
}
|
||||||
|
|
||||||
|
private func buildTonelessIfNeeded() -> [String: [LexiconEntry]] {
|
||||||
|
if let index = tonelessIndex { return index }
|
||||||
|
var best: [String: [String: Double]] = [:]
|
||||||
|
for (reading, entries) in exact {
|
||||||
|
let key = Lexicon.stripTones(reading)
|
||||||
|
for e in entries {
|
||||||
|
if let s = best[key]?[e.phrase], s >= e.score { continue }
|
||||||
|
best[key, default: [:]][e.phrase] = e.score
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
var index: [String: [LexiconEntry]] = [:]
|
||||||
|
index.reserveCapacity(best.count)
|
||||||
|
for (key, phrases) in best {
|
||||||
|
index[key] = phrases.map { LexiconEntry(phrase: $0.key, score: $0.value) }
|
||||||
|
.sorted { $0.score != $1.score ? $0.score > $1.score : $0.phrase < $1.phrase }
|
||||||
|
}
|
||||||
|
tonelessIndex = index
|
||||||
|
return index
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Entries for a sequence of readings. Untoned syllables match any tone.
|
/// Entries for a sequence of readings. Untoned syllables match any tone.
|
||||||
public func entries(for syllables: [Syllable]) -> [LexiconEntry] {
|
public func entries(for syllables: [Syllable]) -> [LexiconEntry] {
|
||||||
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
if syllables.allSatisfy({ $0.tone != nil }) {
|
if syllables.allSatisfy({ $0.tone != nil }) {
|
||||||
return exact[syllables.map(\.reading).joined(separator: "-")] ?? []
|
return exact[syllables.map(\.reading).joined(separator: "-")] ?? []
|
||||||
}
|
}
|
||||||
return toneless[syllables.map(\.toneless).joined(separator: "-")] ?? []
|
return buildTonelessIfNeeded()[syllables.map(\.toneless).joined(separator: "-")] ?? []
|
||||||
}
|
}
|
||||||
|
|
||||||
public func bestScore(for syllable: Syllable) -> Double? {
|
public func bestScore(for syllable: Syllable) -> Double? {
|
||||||
@@ -81,4 +182,13 @@ public final class Lexicon {
|
|||||||
guard let first = s.unicodeScalars.first else { return false }
|
guard let first = s.unicodeScalars.first else { return false }
|
||||||
return (0x3105...0x312F).contains(first.value) || first == "˙"
|
return (0x3105...0x312F).contains(first.value) || first == "˙"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Same test on raw UTF-8: Bopomofo U+3105–U+312F is E3 84 85…E3 84 AF; "˙" is CB 99.
|
||||||
|
private static func isBopomofo(_ b: UnsafeBufferPointer<UInt8>, _ r: Range<Int>) -> Bool {
|
||||||
|
guard r.count >= 2 else { return false }
|
||||||
|
if b[r.lowerBound] == 0xCB && b[r.lowerBound + 1] == 0x99 { return true }
|
||||||
|
guard r.count >= 3, b[r.lowerBound] == 0xE3, b[r.lowerBound + 1] == 0x84 else { return false }
|
||||||
|
let c = b[r.lowerBound + 2]
|
||||||
|
return c >= 0x85 && c <= 0xAF
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,7 +6,10 @@ private let resources = URL(fileURLWithPath: #filePath)
|
|||||||
.appendingPathComponent("Resources")
|
.appendingPathComponent("Resources")
|
||||||
|
|
||||||
private let sharedLexicon = try! Lexicon(contentsOf: resources.appendingPathComponent("lexicon.txt"))
|
private let sharedLexicon = try! Lexicon(contentsOf: resources.appendingPathComponent("lexicon.txt"))
|
||||||
private let sharedEnglish = EnglishModel(commonWordsURL: resources.appendingPathComponent("english-common.txt"))
|
private let windowsData = resources.deletingLastPathComponent().appendingPathComponent("windows/autozhuyin/data")
|
||||||
|
private let sharedEnglish = EnglishModel(commonWordsURL: resources.appendingPathComponent("english-common.txt"),
|
||||||
|
dictionaryURL: windowsData.appendingPathComponent("english-dict.txt"),
|
||||||
|
bigramURL: windowsData.appendingPathComponent("english-bigram.json"))
|
||||||
private let sharedWeights = Features.loadWeights(from: resources.appendingPathComponent("classifier-weights.json"))
|
private let sharedWeights = Features.loadWeights(from: resources.appendingPathComponent("classifier-weights.json"))
|
||||||
|
|
||||||
final class ParserTests: XCTestCase {
|
final class ParserTests: XCTestCase {
|
||||||
@@ -265,10 +268,13 @@ final class ParityTests: XCTestCase {
|
|||||||
for fx in fixtures {
|
for fx in fixtures {
|
||||||
let keys = Array(fx["keys"] as! String)
|
let keys = Array(fx["keys"] as! String)
|
||||||
let expected = (fx["segments"] as! [[Any]]).map { "\($0[0])-\($0[1]):\($0[2])" }
|
let expected = (fx["segments"] as! [[Any]]).map { "\($0[0])-\($0[1]):\($0[2])" }
|
||||||
let got = c.segment(keys, finishing: fx["finishing"] as! Bool).map {
|
let previous = (fx["previous"] as? String).flatMap(InputMode.init(rawValue:))
|
||||||
|
let got = c.segment(keys, finishing: fx["finishing"] as! Bool, previous: previous).map {
|
||||||
"\($0.range.lowerBound)-\($0.range.upperBound):\($0.mode.rawValue)"
|
"\($0.range.lowerBound)-\($0.range.upperBound):\($0.mode.rawValue)"
|
||||||
}
|
}
|
||||||
if got != expected { mismatches.append("\(String(keys)) finishing=\(fx["finishing"]!): \(got) vs \(expected)") }
|
if got != expected {
|
||||||
|
mismatches.append("\(String(keys)) finishing=\(fx["finishing"]!) previous=\(String(describing: previous)): \(got) vs \(expected)")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
XCTAssertGreaterThan(fixtures.count, 300)
|
XCTAssertGreaterThan(fixtures.count, 300)
|
||||||
XCTAssertTrue(mismatches.isEmpty, "\(mismatches.count) mismatches:\n" + mismatches.prefix(10).joined(separator: "\n"))
|
XCTAssertTrue(mismatches.isEmpty, "\(mismatches.count) mismatches:\n" + mismatches.prefix(10).joined(separator: "\n"))
|
||||||
@@ -297,13 +303,16 @@ final class ShiftTapTests: XCTestCase {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
func testShiftTapForcesNextWordEnglishLowercase() {
|
func testShiftTapTogglesEnglishMode() {
|
||||||
let e = engine()
|
let e = engine()
|
||||||
// "ru" alone would be ㄐㄧ (機); after a Shift tap it stays English, lowercase.
|
// "ru" alone would be ㄐㄧ (機); in English mode keys pass straight through, lowercase.
|
||||||
XCTAssertEqual(type("⇧ru ", into: e), "ru ")
|
XCTAssertEqual(type("⇧ru su ", into: e), "ru su ")
|
||||||
// Automatic detection resumes after the word.
|
XCTAssertTrue(e.isEnglishMode) // still English after spaces
|
||||||
XCTAssertFalse(e.isEnglishArmed)
|
XCTAssertEqual(e.markedText, "") // never composed: no underline
|
||||||
XCTAssertEqual(type("su3cl3\n", into: e), "你好")
|
XCTAssertEqual(type("\n", into: e), "\n") // Enter goes to the app, mode stays
|
||||||
|
XCTAssertTrue(e.isEnglishMode)
|
||||||
|
XCTAssertEqual(type("⇧su3cl3\n", into: e), "你好") // tap again: automatic again
|
||||||
|
XCTAssertFalse(e.isEnglishMode)
|
||||||
}
|
}
|
||||||
|
|
||||||
func testShiftTapTwiceCancels() {
|
func testShiftTapTwiceCancels() {
|
||||||
@@ -316,22 +325,22 @@ final class ShiftTapTests: XCTestCase {
|
|||||||
XCTAssertEqual(e.mode, .literal)
|
XCTAssertEqual(e.mode, .literal)
|
||||||
_ = e.handle(.shiftTap)
|
_ = e.handle(.shiftTap)
|
||||||
XCTAssertEqual(e.mode, .zhuyin)
|
XCTAssertEqual(e.mode, .zhuyin)
|
||||||
|
XCTAssertFalse(e.isEnglishMode) // a flip, not English mode
|
||||||
_ = type("\n", into: e)
|
_ = type("\n", into: e)
|
||||||
XCTAssertEqual(e.classifier.learned.weights["so"], 1) // a correction is learned
|
XCTAssertEqual(e.classifier.learned.weights["so"], 1) // a correction is learned
|
||||||
}
|
}
|
||||||
|
|
||||||
func testForcingBeforeTypingIsNotLearned() {
|
func testEnglishModeIsNotLearned() {
|
||||||
let e = engine()
|
let e = engine()
|
||||||
_ = type("⇧hello ", into: e)
|
_ = type("⇧hello ", into: e)
|
||||||
XCTAssertNil(e.classifier.learned.weights["hello"])
|
XCTAssertTrue(e.classifier.learned.weights.isEmpty)
|
||||||
}
|
}
|
||||||
|
|
||||||
func testArmingSurvivesBackspace() {
|
func testBackspaceInEnglishModeGoesToApp() {
|
||||||
let e = engine()
|
let e = engine()
|
||||||
_ = type("⇧r", into: e)
|
_ = type("⇧", into: e)
|
||||||
_ = e.handle(.backspace)
|
XCTAssertFalse(e.handle(.backspace).handled)
|
||||||
XCTAssertTrue(e.isEnglishArmed)
|
XCTAssertTrue(e.isEnglishMode)
|
||||||
XCTAssertEqual(type("ru ", into: e), "ru ")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func testDetector() {
|
func testDetector() {
|
||||||
@@ -406,26 +415,27 @@ final class CursorTests: XCTestCase {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
final class DirectEnglishTests: XCTestCase {
|
final class FirstToneAfterEnglishTests: XCTestCase {
|
||||||
func testForcedEnglishHasNoComposition() {
|
/// After an English word, a first-tone syllable + space must stay Chinese (機, 吃, 書…).
|
||||||
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
func testFirstToneAfterEnglish() {
|
||||||
_ = e.handle(.shiftTap)
|
for keys in ["ru", "t", "gj", "vup", "tj", "el"] {
|
||||||
let r = e.handle(.character("r"))
|
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||||
XCTAssertEqual(r.commit, "r") // inserted right away
|
var out = ""
|
||||||
XCTAssertEqual(e.markedText, "") // nothing underlined
|
for c in "hello " + keys + " " { if let t = e.handle(.character(c)).commit { out += t } }
|
||||||
XCTAssertEqual(e.handle(.character("u")).commit, "u")
|
XCTAssertEqual(out, "hello ", keys)
|
||||||
XCTAssertEqual(e.handle(.character("'")).commit, "'")
|
XCTAssertEqual(e.markedText.count, 1, keys) // still composing one Chinese character
|
||||||
XCTAssertFalse(e.handle(.backspace).handled) // the app deletes it
|
}
|
||||||
XCTAssertTrue(e.isEnglishArmed)
|
|
||||||
XCTAssertEqual(e.handle(.character(" ")).commit, " ")
|
|
||||||
XCTAssertFalse(e.isEnglishArmed) // back to automatic
|
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func testEnterEndsForcedEnglish() {
|
final class StartupTimingTests: XCTestCase {
|
||||||
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
func testStartupTiming() throws {
|
||||||
_ = e.handle(.shiftTap)
|
var t = Date()
|
||||||
_ = e.handle(.character("a"))
|
func lap(_ name: String) { print(String(format: "STARTUP %@: %.3fs", name, Date().timeIntervalSince(t))); t = Date() }
|
||||||
XCTAssertFalse(e.handle(.enter).handled)
|
_ = try Lexicon(contentsOf: resources.appendingPathComponent("lexicon.txt")); lap("lexicon")
|
||||||
XCTAssertFalse(e.isEnglishArmed)
|
_ = EnglishModel(commonWordsURL: resources.appendingPathComponent("english-common.txt"),
|
||||||
|
dictionaryURL: windowsData.appendingPathComponent("english-dict.txt"),
|
||||||
|
bigramURL: windowsData.appendingPathComponent("english-bigram.json")); lap("english")
|
||||||
|
_ = Features.loadWeights(from: resources.appendingPathComponent("classifier-weights.json")); lap("weights")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,6 +19,9 @@ mkdir -p "$APP/Contents/MacOS" "$APP/Contents/Resources"
|
|||||||
cp "$BIN" "$APP/Contents/MacOS/AutoZhuyin"
|
cp "$BIN" "$APP/Contents/MacOS/AutoZhuyin"
|
||||||
cp Resources/Info.plist "$APP/Contents/Info.plist"
|
cp Resources/Info.plist "$APP/Contents/Info.plist"
|
||||||
cp Resources/lexicon.txt Resources/english-common.txt Resources/classifier-weights.json "$APP/Contents/Resources/"
|
cp Resources/lexicon.txt Resources/english-common.txt Resources/classifier-weights.json "$APP/Contents/Resources/"
|
||||||
|
# Precomputed English dictionary and bigram table (shared with the Windows build).
|
||||||
|
python3 -I windows/tools/build_data.py >/dev/null
|
||||||
|
cp windows/autozhuyin/data/english-dict.txt windows/autozhuyin/data/english-bigram.json "$APP/Contents/Resources/"
|
||||||
|
|
||||||
if [ ! -f build/icon.tiff ]; then
|
if [ ! -f build/icon.tiff ]; then
|
||||||
xcrun swift scripts/make-icon.swift build/icon.tiff
|
xcrun swift scripts/make-icon.swift build/icon.tiff
|
||||||
|
|||||||
@@ -23,14 +23,14 @@ def main():
|
|||||||
clf = core.Classifier(lexicon, english, weights=core.Classifier.load_weights(WEIGHTS_PATH))
|
clf = core.Classifier(lexicon, english, weights=core.Classifier.load_weights(WEIGHTS_PATH))
|
||||||
gen = Generator(os.path.join(DATA, "lexicon.txt"), os.path.join(DATA, "english-common.txt"),
|
gen = Generator(os.path.join(DATA, "lexicon.txt"), os.path.join(DATA, "english-common.txt"),
|
||||||
os.path.join(DATA, "english-dict.txt"), seed=7)
|
os.path.join(DATA, "english-dict.txt"), seed=7)
|
||||||
cases = [(k, f) for k in HAND_PICKED for f in (False, True)]
|
cases = [(k, f, p) for k in HAND_PICKED for f in (False, True) for p in (None, "literal")]
|
||||||
while len(cases) < 400:
|
while len(cases) < 500:
|
||||||
ex = gen.example()
|
ex = gen.example()
|
||||||
if len(ex["keys"]) <= 28:
|
if len(ex["keys"]) <= 28:
|
||||||
cases.append((ex["keys"], ex["finishing"]))
|
cases.append((ex["keys"], ex["finishing"], ex["previous"]))
|
||||||
cases.append((gen.prefix_of(ex)["keys"], False))
|
cases.append((gen.prefix_of(ex)["keys"], False, ex["previous"]))
|
||||||
fixtures = [{"keys": k, "finishing": f,
|
fixtures = [{"keys": k, "finishing": f, "previous": p,
|
||||||
"segments": [[s, e, m] for s, e, m in clf.segment(list(k), f)]} for k, f in cases]
|
"segments": [[s, e, m] for s, e, m in clf.segment(list(k), f, p)]} for k, f, p in cases]
|
||||||
with open(OUT, "w", encoding="utf-8") as fh:
|
with open(OUT, "w", encoding="utf-8") as fh:
|
||||||
json.dump(fixtures, fh, ensure_ascii=False, indent=0)
|
json.dump(fixtures, fh, ensure_ascii=False, indent=0)
|
||||||
print("wrote %d fixtures to %s" % (len(fixtures), OUT))
|
print("wrote %d fixtures to %s" % (len(fixtures), OUT))
|
||||||
|
|||||||
+5114
-3374
File diff suppressed because it is too large
Load Diff
+17
-2
@@ -102,16 +102,31 @@ class Generator:
|
|||||||
labels = "".join(m * len(p) for m, p in parts)
|
labels = "".join(m * len(p) for m, p in parts)
|
||||||
if parts[-1][0] == "l":
|
if parts[-1][0] == "l":
|
||||||
finishing = True # English / numbers end with a space
|
finishing = True # English / numbers end with a space
|
||||||
|
elif keys[-1] == " " and self.rng.random() < 0.6:
|
||||||
|
# The moment the space (first tone) is pressed: the buffer has no space yet and
|
||||||
|
# the input method decides "first tone" vs "English word + space".
|
||||||
|
keys, labels, finishing = keys[:-1], labels[:-1], True
|
||||||
|
kind += "/tone1"
|
||||||
else:
|
else:
|
||||||
# Chinese ends with Enter, or space after a finished (toned) syllable.
|
# Chinese ends with Enter, or space after a finished (toned) syllable.
|
||||||
finishing = keys[-1] != " " and self.rng.random() < 0.5
|
finishing = keys[-1] != " " and self.rng.random() < 0.5
|
||||||
return {"keys": keys, "labels": labels, "finishing": finishing, "kind": kind}
|
return {"keys": keys, "labels": labels, "finishing": finishing, "kind": kind,
|
||||||
|
"previous": self.previous_mode(labels[0])}
|
||||||
|
|
||||||
|
def previous_mode(self, first_label):
|
||||||
|
"""Mode of the text committed just before: none, mostly the same, sometimes the other."""
|
||||||
|
r = self.rng.random()
|
||||||
|
if r < 0.35:
|
||||||
|
return None
|
||||||
|
same = core.ZHUYIN if first_label == "z" else core.LITERAL
|
||||||
|
other = core.LITERAL if first_label == "z" else core.ZHUYIN
|
||||||
|
return same if r < 0.80 else other
|
||||||
|
|
||||||
def prefix_of(self, ex):
|
def prefix_of(self, ex):
|
||||||
"""The buffer while still typing: a random cut, not finishing."""
|
"""The buffer while still typing: a random cut, not finishing."""
|
||||||
cut = self.rng.randint(1, len(ex["keys"]) - 1) if len(ex["keys"]) > 1 else 1
|
cut = self.rng.randint(1, len(ex["keys"]) - 1) if len(ex["keys"]) > 1 else 1
|
||||||
return {"keys": ex["keys"][:cut], "labels": ex["labels"][:cut], "finishing": False,
|
return {"keys": ex["keys"][:cut], "labels": ex["labels"][:cut], "finishing": False,
|
||||||
"kind": ex["kind"] + "/prefix"}
|
"kind": ex["kind"] + "/prefix", "previous": ex["previous"]}
|
||||||
|
|
||||||
|
|
||||||
def _cumulative(ws):
|
def _cumulative(ws):
|
||||||
|
|||||||
+19
-10
@@ -45,12 +45,12 @@ def labels_of(segs, n):
|
|||||||
return "".join(lab)
|
return "".join(lab)
|
||||||
|
|
||||||
|
|
||||||
def seg_features(clf, keys, segs, finishing):
|
def seg_features(clf, keys, segs, finishing, previous):
|
||||||
"""Summed features of a segmentation, or None if any segment is impossible."""
|
"""Summed features of a segmentation, or None if any segment is impossible."""
|
||||||
total = defaultdict(float)
|
total = defaultdict(float)
|
||||||
n = len(keys)
|
n = len(keys)
|
||||||
for idx, (s, e, m) in enumerate(segs):
|
for idx, (s, e, m) in enumerate(segs):
|
||||||
fz, fl = clf.segment_features(keys[s:e], e == n, idx > 0, finishing)
|
fz, fl = clf.segment_features(keys[s:e], e == n, idx > 0, finishing, previous if idx == 0 else None)
|
||||||
f = fz if m == core.ZHUYIN else fl
|
f = fz if m == core.ZHUYIN else fl
|
||||||
if f is None:
|
if f is None:
|
||||||
return None
|
return None
|
||||||
@@ -62,12 +62,13 @@ def seg_features(clf, keys, segs, finishing):
|
|||||||
def evaluate(clf, weights, data):
|
def evaluate(clf, weights, data):
|
||||||
by_kind = defaultdict(lambda: [0, 0])
|
by_kind = defaultdict(lambda: [0, 0])
|
||||||
for ex in data:
|
for ex in data:
|
||||||
segs = clf.segment(list(ex["keys"]), ex["finishing"], None, weights)
|
segs = clf.segment(list(ex["keys"]), ex["finishing"], ex["previous"], weights)
|
||||||
ok = labels_of(segs, len(ex["keys"])) == ex["labels"]
|
ok = labels_of(segs, len(ex["keys"])) == ex["labels"]
|
||||||
bucket = by_kind["prefix" if ex["kind"].endswith("/prefix") else ex["kind"]]
|
kind = "prefix" if ex["kind"].endswith("/prefix") else ex["kind"]
|
||||||
bucket[0] += ok
|
for k in (kind, "after-other") if _after_other(ex) else (kind,):
|
||||||
bucket[1] += 1
|
by_kind[k][0] += ok
|
||||||
commit = [v for k, v in by_kind.items() if k != "prefix"]
|
by_kind[k][1] += 1
|
||||||
|
commit = [v for k, v in by_kind.items() if k not in ("prefix", "after-other")]
|
||||||
summary = {
|
summary = {
|
||||||
"commit": sum(c for c, _ in commit) / max(1, sum(t for _, t in commit)),
|
"commit": sum(c for c, _ in commit) / max(1, sum(t for _, t in commit)),
|
||||||
"prefix": by_kind["prefix"][0] / max(1, by_kind["prefix"][1]),
|
"prefix": by_kind["prefix"][0] / max(1, by_kind["prefix"][1]),
|
||||||
@@ -76,6 +77,14 @@ def evaluate(clf, weights, data):
|
|||||||
return summary
|
return summary
|
||||||
|
|
||||||
|
|
||||||
|
def _after_other(ex):
|
||||||
|
"""Committed buffer whose first segment differs from the text typed just before."""
|
||||||
|
if ex["kind"].endswith("/prefix") or ex["previous"] is None:
|
||||||
|
return False
|
||||||
|
first = core.ZHUYIN if ex["labels"][0] == "z" else core.LITERAL
|
||||||
|
return ex["previous"] != first
|
||||||
|
|
||||||
|
|
||||||
def show(name, s):
|
def show(name, s):
|
||||||
kinds = " ".join("%s %.1f%%" % (k, 100 * v) for k, v in s["by_kind"].items())
|
kinds = " ".join("%s %.1f%%" % (k, 100 * v) for k, v in s["by_kind"].items())
|
||||||
print("%-9s commit %.2f%% typing %.2f%% [%s]" % (name, 100 * s["commit"], 100 * s["prefix"], kinds))
|
print("%-9s commit %.2f%% typing %.2f%% [%s]" % (name, 100 * s["commit"], 100 * s["prefix"], kinds))
|
||||||
@@ -92,11 +101,11 @@ def train(clf, data, epochs, lr, seed):
|
|||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
for ex in data:
|
for ex in data:
|
||||||
keys = list(ex["keys"])
|
keys = list(ex["keys"])
|
||||||
pred = clf.segment(keys, ex["finishing"], None, w)
|
pred = clf.segment(keys, ex["finishing"], ex["previous"], w)
|
||||||
if labels_of(pred, len(keys)) != ex["labels"]:
|
if labels_of(pred, len(keys)) != ex["labels"]:
|
||||||
gold = gold_segments(ex["labels"])
|
gold = gold_segments(ex["labels"])
|
||||||
fg = seg_features(clf, keys, gold, ex["finishing"])
|
fg = seg_features(clf, keys, gold, ex["finishing"], ex["previous"])
|
||||||
fp = seg_features(clf, keys, pred, ex["finishing"])
|
fp = seg_features(clf, keys, pred, ex["finishing"], ex["previous"])
|
||||||
if fg is not None and fp is not None:
|
if fg is not None and fp is not None:
|
||||||
mistakes += 1
|
mistakes += 1
|
||||||
scale = lr * (0.5 if ex["kind"].endswith("/prefix") else 1.0)
|
scale = lr * (0.5 if ex["kind"].endswith("/prefix") else 1.0)
|
||||||
|
|||||||
@@ -142,8 +142,8 @@ class AutoZhuyinTextService(TextService):
|
|||||||
return False
|
return False
|
||||||
self.engine.handle(SHIFT_TAP)
|
self.engine.handle(SHIFT_TAP)
|
||||||
self._update(None)
|
self._update(None)
|
||||||
if self.engine.english_armed:
|
if not self.engine.keys:
|
||||||
self.showMessage("英文", 1)
|
self.showMessage("英文" if self.engine.english_mode else "中英自動", 1)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
# --- replies ---
|
# --- replies ---
|
||||||
|
|||||||
@@ -30,7 +30,6 @@ class LearnedPreferences:
|
|||||||
|
|
||||||
|
|
||||||
class Classifier:
|
class Classifier:
|
||||||
context_weight = 1.5
|
|
||||||
|
|
||||||
def __init__(self, lexicon, english, learned=None, weights=None):
|
def __init__(self, lexicon, english, learned=None, weights=None):
|
||||||
self.lexicon = lexicon
|
self.lexicon = lexicon
|
||||||
@@ -49,20 +48,17 @@ class Classifier:
|
|||||||
except (OSError, ValueError):
|
except (OSError, ValueError):
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
def segment_features(self, seg, is_last, after_other, finishing):
|
def segment_features(self, seg, is_last, after_other, finishing, previous=None):
|
||||||
"""(zhuyin features or None, literal features or None) for one segment."""
|
"""(zhuyin features or None, literal features or None) for one segment.
|
||||||
|
`previous` (mode of the text committed just before) only affects the first segment."""
|
||||||
seg = list(seg)
|
seg = list(seg)
|
||||||
return (zhuyin_features(seg, self.lexicon, is_last, after_other, finishing),
|
return (zhuyin_features(seg, self.lexicon, is_last, after_other, finishing, previous),
|
||||||
literal_features(seg, self.english, is_last, after_other, finishing))
|
literal_features(seg, self.english, is_last, after_other, finishing, previous))
|
||||||
|
|
||||||
def _scores(self, seg, is_last, after_other, finishing, previous, whole, weights):
|
def _scores(self, seg, is_last, after_other, finishing, previous, whole, weights):
|
||||||
fz, fl = self.segment_features(seg, is_last, after_other, finishing)
|
fz, fl = self.segment_features(seg, is_last, after_other, finishing, previous)
|
||||||
zh = dot(weights, fz) if fz is not None else NEG_INF
|
zh = dot(weights, fz) if fz is not None else NEG_INF
|
||||||
lit = dot(weights, fl) if fl is not None else NEG_INF
|
lit = dot(weights, fl) if fl is not None else NEG_INF
|
||||||
if previous == ZHUYIN:
|
|
||||||
zh += self.context_weight
|
|
||||||
elif previous == LITERAL:
|
|
||||||
lit += self.context_weight
|
|
||||||
if whole:
|
if whole:
|
||||||
b = self.learned.bias("".join(seg))
|
b = self.learned.bias("".join(seg))
|
||||||
if b > 0:
|
if b > 0:
|
||||||
|
|||||||
@@ -34,9 +34,9 @@ class CompositionEngine:
|
|||||||
self.now = now
|
self.now = now
|
||||||
self.keys = []
|
self.keys = []
|
||||||
self.forced_mode = None
|
self.forced_mode = None
|
||||||
# True when forced_mode corrects the classifier (learned on commit); False when the
|
# English mode, toggled by a Shift tap when nothing is being typed: keys go straight
|
||||||
# user forced English before typing (Shift tap on an empty buffer).
|
# to the app (no composition, no underline) until Shift is tapped again.
|
||||||
self._forced_is_correction = False
|
self.english_mode = False
|
||||||
self.fixed = {} # zhuyin segment start -> {(syl_start, syl_end): text}
|
self.fixed = {} # zhuyin segment start -> {(syl_start, syl_end): text}
|
||||||
self.candidates = []
|
self.candidates = []
|
||||||
self.candidate_index = 0
|
self.candidate_index = 0
|
||||||
@@ -150,26 +150,16 @@ class CompositionEngine:
|
|||||||
|
|
||||||
# --- key handling ---
|
# --- key handling ---
|
||||||
|
|
||||||
@property
|
|
||||||
def english_armed(self):
|
|
||||||
"""True while the user has forced English for the next word (Shift tap). In this
|
|
||||||
mode keys go straight to the app with no composition (no underline)."""
|
|
||||||
return not self.keys and self.forced_mode == LITERAL and not self._forced_is_correction
|
|
||||||
|
|
||||||
def _end_direct_english(self):
|
|
||||||
if self.english_armed:
|
|
||||||
self.forced_mode = None
|
|
||||||
self._previous_mode = LITERAL
|
|
||||||
self._previous_time = self.now()
|
|
||||||
|
|
||||||
def _flip(self):
|
def _flip(self):
|
||||||
|
"""Forces the whole buffer into the other mode (a correction that is learned)."""
|
||||||
self.forced_mode = LITERAL if self.mode == ZHUYIN else ZHUYIN
|
self.forced_mode = LITERAL if self.mode == ZHUYIN else ZHUYIN
|
||||||
self._forced_is_correction = True
|
|
||||||
|
|
||||||
def wants(self, key):
|
def wants(self, key):
|
||||||
"""Whether handle(key) would consume the key, without changing state."""
|
"""Whether handle(key) would consume the key, without changing state."""
|
||||||
if self.candidates or key == SHIFT_TAP or self.english_armed:
|
if self.candidates or key == SHIFT_TAP:
|
||||||
return True
|
return True
|
||||||
|
if self.english_mode and not self.keys:
|
||||||
|
return False # English mode: everything goes to the app as typed
|
||||||
if key in SPECIAL_KEYS:
|
if key in SPECIAL_KEYS:
|
||||||
return bool(self.keys)
|
return bool(self.keys)
|
||||||
if self.keys:
|
if self.keys:
|
||||||
@@ -190,34 +180,29 @@ class CompositionEngine:
|
|||||||
self.keys.pop()
|
self.keys.pop()
|
||||||
self.fixed = {k: v for k, v in self.fixed.items() if k < len(self.keys)}
|
self.fixed = {k: v for k, v in self.fixed.items() if k < len(self.keys)}
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
# A Shift-tap "next word is English" stays armed; a correction is dropped.
|
self.forced_mode = None
|
||||||
if self._forced_is_correction:
|
|
||||||
self.forced_mode = None
|
|
||||||
self._forced_is_correction = False
|
|
||||||
self.fixed = {}
|
self.fixed = {}
|
||||||
return CONSUMED
|
return CONSUMED
|
||||||
if key == ENTER:
|
if key == ENTER:
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
self._end_direct_english()
|
|
||||||
return PASS
|
return PASS
|
||||||
return True, self._commit()
|
return True, self._commit()
|
||||||
if key == ESCAPE:
|
if key == ESCAPE:
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
self._end_direct_english()
|
|
||||||
return PASS
|
return PASS
|
||||||
self.reset()
|
self.reset()
|
||||||
return CONSUMED
|
return CONSUMED
|
||||||
if key == TAB:
|
if key == TAB:
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
self._end_direct_english()
|
|
||||||
return PASS
|
return PASS
|
||||||
self._flip()
|
self._flip()
|
||||||
return CONSUMED
|
return CONSUMED
|
||||||
if key == SHIFT_TAP:
|
if key == SHIFT_TAP:
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
# Next word is English, typed as-is. Tap again to cancel.
|
# Toggle English mode; it stays on until the next Shift tap.
|
||||||
self.forced_mode = None if self.forced_mode == LITERAL else LITERAL
|
self.english_mode = not self.english_mode
|
||||||
self._forced_is_correction = False
|
self._previous_mode = LITERAL if self.english_mode else None
|
||||||
|
self._previous_time = self.now()
|
||||||
else:
|
else:
|
||||||
self._flip()
|
self._flip()
|
||||||
return CONSUMED
|
return CONSUMED
|
||||||
@@ -243,15 +228,8 @@ class CompositionEngine:
|
|||||||
|
|
||||||
def _handle_character(self, c):
|
def _handle_character(self, c):
|
||||||
self.cursor = None # typing always continues at the end
|
self.cursor = None # typing always continues at the end
|
||||||
if self.english_armed:
|
if self.english_mode and not self.keys:
|
||||||
# Forced English: insert directly. Space ends the word; other symbols end it too.
|
return PASS # English mode: every key goes to the app as typed
|
||||||
if c == " ":
|
|
||||||
self._end_direct_english()
|
|
||||||
return True, " "
|
|
||||||
if c.isascii() and (c.isalnum() or c in "'.,-/;"):
|
|
||||||
return True, c
|
|
||||||
self._end_direct_english()
|
|
||||||
return PASS
|
|
||||||
if c == " ":
|
if c == " ":
|
||||||
if not self.keys:
|
if not self.keys:
|
||||||
return PASS
|
return PASS
|
||||||
@@ -329,7 +307,7 @@ class CompositionEngine:
|
|||||||
def _commit(self, segs=None):
|
def _commit(self, segs=None):
|
||||||
segs = segs if segs is not None else self.segments
|
segs = segs if segs is not None else self.segments
|
||||||
text = "".join(self._render(s, True) for s in segs)
|
text = "".join(self._render(s, True) for s in segs)
|
||||||
if self.forced_mode and self._forced_is_correction:
|
if self.forced_mode:
|
||||||
self.classifier.learned.record("".join(self.keys), self.forced_mode)
|
self.classifier.learned.record("".join(self.keys), self.forced_mode)
|
||||||
self._previous_mode = segs[-1][2] if segs else None
|
self._previous_mode = segs[-1][2] if segs else None
|
||||||
self._previous_time = self.now()
|
self._previous_time = self.now()
|
||||||
@@ -343,6 +321,5 @@ class CompositionEngine:
|
|||||||
self.keys = []
|
self.keys = []
|
||||||
self.cursor = None
|
self.cursor = None
|
||||||
self.forced_mode = None
|
self.forced_mode = None
|
||||||
self._forced_is_correction = False
|
|
||||||
self.fixed = {}
|
self.fixed = {}
|
||||||
self.candidates = []
|
self.candidates = []
|
||||||
|
|||||||
@@ -41,6 +41,9 @@ DEFAULT_WEIGHTS = {
|
|||||||
"lit.others": -1.5,
|
"lit.others": -1.5,
|
||||||
# Segmentation
|
# Segmentation
|
||||||
"seg.switch": -3.0,
|
"seg.switch": -3.0,
|
||||||
|
# Context: the first segment has the same mode as the previously committed text
|
||||||
|
"ctx.zh": 1.5,
|
||||||
|
"ctx.lit": 1.5,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -52,7 +55,7 @@ def _add(f, k, v=1.0):
|
|||||||
f[k] = f.get(k, 0.0) + v
|
f[k] = f.get(k, 0.0) + v
|
||||||
|
|
||||||
|
|
||||||
def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=False):
|
def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=False, previous=None):
|
||||||
"""Features of `keys` read as Zhuyin, or None if impossible."""
|
"""Features of `keys` read as Zhuyin, or None if impossible."""
|
||||||
parse = bopomofo.parse(keys)
|
parse = bopomofo.parse(keys)
|
||||||
if not is_last and parse.pending is not None:
|
if not is_last and parse.pending is not None:
|
||||||
@@ -99,10 +102,12 @@ def zhuyin_features(keys, lexicon, is_last=True, after_other=False, finishing=Fa
|
|||||||
_add(f, "zh.unconfirmed")
|
_add(f, "zh.unconfirmed")
|
||||||
if after_other:
|
if after_other:
|
||||||
_add(f, "seg.switch")
|
_add(f, "seg.switch")
|
||||||
|
elif previous == "zhuyin":
|
||||||
|
_add(f, "ctx.zh")
|
||||||
return f
|
return f
|
||||||
|
|
||||||
|
|
||||||
def literal_features(keys, english, is_last=True, after_other=False, finishing=False):
|
def literal_features(keys, english, is_last=True, after_other=False, finishing=False, previous=None):
|
||||||
"""Features of `keys` output as typed, or None if impossible."""
|
"""Features of `keys` output as typed, or None if impossible."""
|
||||||
if not keys or " " in keys:
|
if not keys or " " in keys:
|
||||||
return None
|
return None
|
||||||
@@ -110,6 +115,8 @@ def literal_features(keys, english, is_last=True, after_other=False, finishing=F
|
|||||||
f = {"lit.bias": 1.0}
|
f = {"lit.bias": 1.0}
|
||||||
if after_other:
|
if after_other:
|
||||||
_add(f, "seg.switch")
|
_add(f, "seg.switch")
|
||||||
|
elif previous == "literal":
|
||||||
|
_add(f, "ctx.lit")
|
||||||
if any(c.isupper() for c in keys):
|
if any(c.isupper() for c in keys):
|
||||||
_add(f, "lit.upper")
|
_add(f, "lit.upper")
|
||||||
letters = "".join(c for c in keys if c.isalpha())
|
letters = "".join(c for c in keys if c.isalpha())
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ class Lexicon:
|
|||||||
|
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.exact = {}
|
self.exact = {}
|
||||||
self.toneless = {}
|
self._toneless = None # built on first use (only untoned syllables need it)
|
||||||
self.max_phrase_length = 1
|
self.max_phrase_length = 1
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -42,9 +42,10 @@ class Lexicon:
|
|||||||
except ValueError:
|
except ValueError:
|
||||||
score = -6.0
|
score = -6.0
|
||||||
self.add(reading, phrase, score)
|
self.add(reading, phrase, score)
|
||||||
for table in (self.exact, self.toneless):
|
for v in self.exact.values():
|
||||||
for v in table.values():
|
if len(v) > 1:
|
||||||
v.sort(key=lambda e: -e[1])
|
v.sort(key=lambda e: -e[1])
|
||||||
|
self._toneless = None
|
||||||
|
|
||||||
def add(self, reading, phrase, score):
|
def add(self, reading, phrase, score):
|
||||||
syllables = [s[1:] + "˙" if s.startswith("˙") else s for s in reading.split("-")]
|
syllables = [s[1:] + "˙" if s.startswith("˙") else s for s in reading.split("-")]
|
||||||
@@ -53,15 +54,20 @@ class Lexicon:
|
|||||||
entries = self.exact.setdefault(reading, [])
|
entries = self.exact.setdefault(reading, [])
|
||||||
if not any(p == phrase for p, _ in entries):
|
if not any(p == phrase for p, _ in entries):
|
||||||
entries.append((phrase, score))
|
entries.append((phrase, score))
|
||||||
key = _strip_tones(reading)
|
self._toneless = None
|
||||||
tl = self.toneless.setdefault(key, [])
|
|
||||||
for i, (p, s) in enumerate(tl):
|
@property
|
||||||
if p == phrase:
|
def toneless(self):
|
||||||
if score > s:
|
"""Index ignoring tones: toneless reading -> [(phrase, best score)], best first."""
|
||||||
tl[i] = (p, score)
|
if self._toneless is None:
|
||||||
break
|
best = {}
|
||||||
else:
|
for reading, entries in self.exact.items():
|
||||||
tl.append((phrase, score))
|
bucket = best.setdefault(_strip_tones(reading), {})
|
||||||
|
for phrase, score in entries:
|
||||||
|
if score > bucket.get(phrase, float("-inf")):
|
||||||
|
bucket[phrase] = score
|
||||||
|
self._toneless = {k: sorted(v.items(), key=lambda e: -e[1]) for k, v in best.items()}
|
||||||
|
return self._toneless
|
||||||
|
|
||||||
def entries(self, syllables):
|
def entries(self, syllables):
|
||||||
if all(s.tone is not None for s in syllables):
|
if all(s.tone is not None for s in syllables):
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "自動注音 AutoZhuyin",
|
"name": "自動注音 AutoZhuyin",
|
||||||
"version": "0.5.1",
|
"version": "0.6.2",
|
||||||
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
||||||
"locale": "zh-Hant-TW",
|
"locale": "zh-Hant-TW",
|
||||||
"fallbackLocale": "zh-TW",
|
"fallbackLocale": "zh-TW",
|
||||||
|
|||||||
@@ -29,9 +29,9 @@
|
|||||||
← → 在組字區移動游標,選要改哪個字
|
← → 在組字區移動游標,選要改哪個字
|
||||||
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字
|
↓ 開啟游標前那個字或詞的候選字,按數字鍵 1–9 選字
|
||||||
Esc 取消組字
|
Esc 取消組字
|
||||||
單按 Shift 下一個字強制英文(保持小寫、直接送出、沒有底線),
|
單按 Shift 切換英文模式(按鍵直接送出、保持小寫、沒有底線),
|
||||||
按空白或 Enter 後恢復自動判斷;
|
再單按一次回到中英自動判斷,會出現「英文」/「中英自動」小提示;
|
||||||
打到一半時按,同 Tab 整段翻轉。會出現「英文」小提示
|
打到一半時按,同 Tab 整段翻轉
|
||||||
Shift+字母 打大寫字母(也會強制英文)
|
Shift+字母 打大寫字母(也會強制英文)
|
||||||
< > ? ! : 中文語境下自動變成全形 ,。?!:
|
< > ? ! : 中文語境下自動變成全形 ,。?!:
|
||||||
|
|
||||||
|
|||||||
+21
-29
@@ -194,11 +194,15 @@ class MixedSegmentTests(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class ShiftTapTests(unittest.TestCase):
|
class ShiftTapTests(unittest.TestCase):
|
||||||
def test_forces_next_word_english_lowercase(self):
|
def test_toggles_english_mode(self):
|
||||||
e = make_engine()
|
e = make_engine()
|
||||||
self.assertEqual(type_keys("⇧ru ", e), "ru ")
|
self.assertEqual(type_keys("⇧ru su ", e), "ru su ")
|
||||||
self.assertFalse(e.english_armed)
|
self.assertTrue(e.english_mode)
|
||||||
self.assertEqual(type_keys("su3cl3\n", e), "你好")
|
self.assertEqual(e.marked_text, "")
|
||||||
|
self.assertEqual(e.handle(ENTER), (False, None))
|
||||||
|
self.assertTrue(e.english_mode)
|
||||||
|
self.assertEqual(type_keys("⇧su3cl3\n", e), "你好")
|
||||||
|
self.assertFalse(e.english_mode)
|
||||||
|
|
||||||
def test_tap_twice_cancels(self):
|
def test_tap_twice_cancels(self):
|
||||||
self.assertEqual(type_keys("⇧⇧su3cl3\n", make_engine()), "你好")
|
self.assertEqual(type_keys("⇧⇧su3cl3\n", make_engine()), "你好")
|
||||||
@@ -208,20 +212,20 @@ class ShiftTapTests(unittest.TestCase):
|
|||||||
type_keys("so", e)
|
type_keys("so", e)
|
||||||
e.handle(SHIFT_TAP)
|
e.handle(SHIFT_TAP)
|
||||||
self.assertEqual(e.mode, ZHUYIN)
|
self.assertEqual(e.mode, ZHUYIN)
|
||||||
|
self.assertFalse(e.english_mode)
|
||||||
type_keys("\n", e)
|
type_keys("\n", e)
|
||||||
self.assertEqual(e.classifier.learned.weights["so"], 1)
|
self.assertEqual(e.classifier.learned.weights["so"], 1)
|
||||||
|
|
||||||
def test_forcing_before_typing_not_learned(self):
|
def test_english_mode_not_learned(self):
|
||||||
e = make_engine()
|
e = make_engine()
|
||||||
type_keys("⇧hello ", e)
|
type_keys("⇧hello ", e)
|
||||||
self.assertNotIn("hello", e.classifier.learned.weights)
|
self.assertEqual(e.classifier.learned.weights, {})
|
||||||
|
|
||||||
def test_arming_survives_backspace(self):
|
def test_backspace_goes_to_app(self):
|
||||||
e = make_engine()
|
e = make_engine()
|
||||||
type_keys("⇧r", e)
|
e.handle(SHIFT_TAP)
|
||||||
e.handle(BACKSPACE)
|
self.assertEqual(e.handle(BACKSPACE), (False, None))
|
||||||
self.assertTrue(e.english_armed)
|
self.assertTrue(e.english_mode)
|
||||||
self.assertEqual(type_keys("ru ", e), "ru ")
|
|
||||||
|
|
||||||
def test_detector(self):
|
def test_detector(self):
|
||||||
d = ShiftTapDetector()
|
d = ShiftTapDetector()
|
||||||
@@ -267,24 +271,12 @@ class CursorTests(unittest.TestCase):
|
|||||||
self.assertTrue(e.showing_candidates)
|
self.assertTrue(e.showing_candidates)
|
||||||
|
|
||||||
|
|
||||||
class DirectEnglishTests(unittest.TestCase):
|
class FirstToneAfterEnglishTests(unittest.TestCase):
|
||||||
def test_forced_english_has_no_composition(self):
|
def test_first_tone_after_english(self):
|
||||||
e = make_engine()
|
for keys in ["ru", "t", "gj", "vup", "tj", "el"]:
|
||||||
e.handle(SHIFT_TAP)
|
e = make_engine()
|
||||||
self.assertEqual(e.handle("r"), (True, "r"))
|
self.assertEqual(type_keys("hello " + keys + " ", e), "hello ", keys)
|
||||||
self.assertEqual(e.marked_text, "")
|
self.assertEqual(len(e.marked_text), 1, keys)
|
||||||
self.assertEqual(e.handle("'"), (True, "'"))
|
|
||||||
self.assertEqual(e.handle(BACKSPACE), (False, None))
|
|
||||||
self.assertTrue(e.english_armed)
|
|
||||||
self.assertEqual(e.handle(" "), (True, " "))
|
|
||||||
self.assertFalse(e.english_armed)
|
|
||||||
|
|
||||||
def test_enter_ends_forced_english(self):
|
|
||||||
e = make_engine()
|
|
||||||
e.handle(SHIFT_TAP)
|
|
||||||
e.handle("a")
|
|
||||||
self.assertEqual(e.handle(ENTER), (False, None))
|
|
||||||
self.assertFalse(e.english_armed)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -144,14 +144,15 @@ class ShiftTapWrapperTests(unittest.TestCase):
|
|||||||
def test_tap_forces_english_lowercase(self):
|
def test_tap_forces_english_lowercase(self):
|
||||||
reply = tap_shift(self.svc)
|
reply = tap_shift(self.svc)
|
||||||
self.assertEqual(reply.get("showMessage"), "英文")
|
self.assertEqual(reply.get("showMessage"), "英文")
|
||||||
self.assertEqual(type_text(self.svc, "ru "), "ru ")
|
self.assertEqual(type_text(self.svc, "ru su "), "ru su ")
|
||||||
|
self.assertEqual(tap_shift(self.svc).get("showMessage"), "中英自動")
|
||||||
|
|
||||||
def test_shift_with_letter_is_not_a_tap(self):
|
def test_shift_with_letter_is_not_a_tap(self):
|
||||||
self.svc.filterKeyDown(FakeKeyEvent("⇧", shift=True))
|
self.svc.filterKeyDown(FakeKeyEvent("⇧", shift=True))
|
||||||
press(self.svc, "H", shift=True)
|
press(self.svc, "H", shift=True)
|
||||||
up = FakeKeyEvent("⇧")
|
up = FakeKeyEvent("⇧")
|
||||||
self.assertFalse(self.svc.filterKeyUp(up))
|
self.assertFalse(self.svc.filterKeyUp(up))
|
||||||
self.assertFalse(self.svc.engine.english_armed)
|
self.assertFalse(self.svc.engine.english_mode)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
Reference in New Issue
Block a user