Learn the previous-word context; fix first-tone characters after English
After an English word, 23 of 50 common first-tone characters (機, 吃, 書…) were read as English when space was pressed: the fixed +1.5 context bonus was far too strong for the trained weight scale, and the training data never contained the "space just pressed on a first-tone syllable" decision. Context is now the ctx.zh / ctx.lit features, trained with the rest; the synthetic data includes the previous mode and the first-tone space decision. Test accuracy: commit 98.22% -> 99.52%, typing 88.73% -> 91.49%, and when the language differs from the previous word 92.0% -> 99.3%. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,39 +1,41 @@
|
||||
{
|
||||
"about": "AutoZhuyin classifier weights, learned by training/train.py",
|
||||
"trained": "2026-11-11",
|
||||
"examples": 40001,
|
||||
"examples": 40000,
|
||||
"test_accuracy": {
|
||||
"commit": 0.9962,
|
||||
"typing": 0.8781
|
||||
"commit": 0.9952,
|
||||
"typing": 0.9149
|
||||
},
|
||||
"weights": {
|
||||
"lit.bias": 0.4276,
|
||||
"lit.common": -0.3465,
|
||||
"lit.common_len": 0.81,
|
||||
"lit.dict": -1.7902,
|
||||
"lit.dict_len": 0.6765,
|
||||
"lit.digits": -2.7403,
|
||||
"lit.num_digits": -0.0751,
|
||||
"lit.num_others": -0.7433,
|
||||
"lit.others": -1.5685,
|
||||
"lit.prefix_common_len": 0.2082,
|
||||
"lit.prefix_dict_len": 0.0003,
|
||||
"lit.short": -0.2397,
|
||||
"lit.unknown_score": 0.213,
|
||||
"ctx.lit": 0.0916,
|
||||
"ctx.zh": 0.1349,
|
||||
"lit.bias": 0.3982,
|
||||
"lit.common": -0.2906,
|
||||
"lit.common_len": 0.8046,
|
||||
"lit.dict": -1.6662,
|
||||
"lit.dict_len": 0.6441,
|
||||
"lit.digits": -2.9689,
|
||||
"lit.num_digits": -0.018,
|
||||
"lit.num_others": -0.5554,
|
||||
"lit.others": -2.0002,
|
||||
"lit.prefix_common_len": 0.2831,
|
||||
"lit.prefix_dict_len": 0.0299,
|
||||
"lit.short": -0.2482,
|
||||
"lit.unknown_score": 0.2479,
|
||||
"lit.upper": 20.0,
|
||||
"seg.switch": -1.7203,
|
||||
"zh.bias": 1.3522,
|
||||
"zh.lexscore": 0.26,
|
||||
"seg.switch": -1.7596,
|
||||
"zh.bias": 1.3423,
|
||||
"zh.lexscore": 0.2917,
|
||||
"zh.pending_invalid": -5.0,
|
||||
"zh.pending_prefix": -0.9295,
|
||||
"zh.phrase": 0.4595,
|
||||
"zh.reordered": -1.4348,
|
||||
"zh.pending_prefix": -0.9162,
|
||||
"zh.phrase": 0.4499,
|
||||
"zh.reordered": -1.3079,
|
||||
"zh.stray": -5.0,
|
||||
"zh.toned_invalid": -5.0,
|
||||
"zh.toned_valid_lex": -0.9823,
|
||||
"zh.toned_valid_nolex": -1.8742,
|
||||
"zh.unconfirmed": -0.7891,
|
||||
"zh.toned_valid_lex": -0.9018,
|
||||
"zh.toned_valid_nolex": -1.6341,
|
||||
"zh.unconfirmed": -0.7952,
|
||||
"zh.untoned_mid_invalid": -5.0,
|
||||
"zh.untoned_mid_valid": -2.8009
|
||||
"zh.untoned_mid_valid": -2.7479
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user