Compare commits
2
Commits
acafd8f96e
...
38e0758a13
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
38e0758a13 | ||
|
|
fd87da8061 |
@@ -17,6 +17,7 @@
|
||||
- **可省略聲調**:`jiap7`(ㄨㄛ ㄇㄣ˙)一樣能打出「我們」
|
||||
- **候選字**:用 ← → 把游標移到要改的字後面,按 ↓ 選字;選過的會固定下來
|
||||
- **記住選字**:從候選字選過的字詞會被記住,下次打同樣的讀音會優先出現(選越多次越優先)
|
||||
- **AI 選字(macOS)**:用中研院 CKIP 的繁中小型語言模型看前後文挑同音字(他跑「得」很快、請你「再」說一次),完全在本機運算;可調整或關閉,系統忙碌時自動暫停
|
||||
- **中間補字、刪字**:游標移到前面後直接打注音,會插在游標處;Backspace 刪掉游標前的那個字
|
||||
- **全形標點**:中文語境下 `?` `!` `<` 自動變成 `?` `!` `,`;任何時候按 `` ` `` 再按標點鍵,一定得到全形標點(例如 `` `< `` = `,`)
|
||||
- **完整詞庫**:約 17 萬筆,來自小麥注音 McBopomofo(MIT 授權)
|
||||
@@ -96,6 +97,37 @@ SmartScreen 出現警告時,按「其他資訊 → 仍要執行」。解除安
|
||||
|
||||
轉換用 Viterbi 演算法,在詞庫中找出整串音節機率最高的詞組組合。
|
||||
|
||||
### AI 選字(macOS)
|
||||
|
||||
詞庫只知道每個詞本身有多常見,不看前後文。AI 選字讓語言模型當裁判:
|
||||
|
||||
1. 詞庫先找出前 K 名的整句組合(例如「請你在說一次」「請你再說一次」)。
|
||||
2. 模型 [CKIP GPT2-tiny](https://huggingface.co/ckiplab/gpt2-tiny-chinese)(中研院詞庫小組,繁體中文,約 1200 萬參數,22 MB)計算每個組合出現的機率,並參考剛送出的前文。
|
||||
3. 分數 = 詞庫分數 + 0.6 × 模型分數,取最高的。選過的候選字仍然固定不變。
|
||||
|
||||
輸入法選單可以調整:
|
||||
|
||||
| 模式 | 比較幾個組合 | 每次運算上限 |
|
||||
|---|---|---|
|
||||
| 關閉 | — | 不載入模型、不佔記憶體 |
|
||||
| 省電 | 3 | 15 毫秒 |
|
||||
| 標準(預設) | 10 | 40 毫秒 |
|
||||
| 高品質 | 30 | 120 毫秒 |
|
||||
|
||||
不會拖慢打字的機制:
|
||||
- 每次運算超過上限就直接放棄,改用詞庫結果;連續 3 次超時就暫停一分鐘。
|
||||
- 「系統忙碌時自動暫停 AI」(預設開啟):Mac 過熱,或 CPU 負載(每核心 1 分鐘平均)超過 0.9 時暫停;低耗電模式下降到省電模式。選單最下面會顯示目前狀態。
|
||||
- 模型在背景載入,載入完成前照常用詞庫。
|
||||
|
||||
實測(`training/lm/eval_rerank.py`,標準模式):
|
||||
|
||||
| 測試資料 | 只用詞庫 | AI 選字 |
|
||||
|---|---|---|
|
||||
| 日常對話 60 句 | 字正確率 96.6%,整句全對 73% | 97.5%,80% |
|
||||
| 維基百科 714 句 | 92.9%,31% | 94.7%,44% |
|
||||
|
||||
錯字大約少了 25–30%。Windows、Linux 版目前還沒有 AI 選字。
|
||||
|
||||
## 個人資料
|
||||
|
||||
| | macOS | Windows | Linux |
|
||||
@@ -190,6 +222,22 @@ python3 training/make_parity.py # 權重或特徵改了之後都要重
|
||||
scripts/import-mcbopomofo.sh
|
||||
```
|
||||
|
||||
### AI 選字模型
|
||||
|
||||
```bash
|
||||
scripts/fetch-lm.sh # 下載 CKIP gpt2-tiny-chinese,轉成 build/lm/lm.bin(float16)+ lm-vocab.txt
|
||||
```
|
||||
|
||||
`build.sh` 會自動執行並把模型打包進 app。模型推論寫在 `Sources/AutoZhuyinCore/LanguageModel.swift`(Accelerate,Apple Silicon 和 Intel 都可用),重排在 `Reranker.swift`。
|
||||
|
||||
評估效果(需要 numpy;`build/lm-eval/` 放測試句子,一行一句):
|
||||
|
||||
```bash
|
||||
python training/lm/eval_rerank.py build/lm-eval/chat.txt build/lm-eval/wiki.txt
|
||||
```
|
||||
|
||||
`training/lm/gpt2.py` 是 numpy 版的參考實作,Swift 版的測試會比對它算出的分數。
|
||||
|
||||
### 發布新版本
|
||||
|
||||
先 commit 所有修改,再執行:
|
||||
@@ -263,7 +311,8 @@ THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝
|
||||
- [x] 用資料訓練判斷權重(合成資料)
|
||||
- [ ] 加入真實打字資料一起訓練
|
||||
- [ ] 已送出的英文單字若判斷錯誤,可以「事後翻轉」
|
||||
- [ ] 用 Core ML 小模型取代手調分數,參考整句語境
|
||||
- [x] AI 選字:語言模型參考前後文挑同音字(macOS)
|
||||
- [ ] AI 選字移植到 Windows、Linux
|
||||
- [x] 使用者詞頻學習(選過的候選字優先)
|
||||
- [ ] 偏好設定視窗
|
||||
|
||||
@@ -272,5 +321,6 @@ THIRD_PARTY_LICENSES.txt 第三方授權聲明(會打包進兩個安裝
|
||||
- 詞庫資料:[小麥注音 McBopomofo](https://github.com/openvanilla/McBopomofo)(MIT License)
|
||||
- Windows 輸入法框架:[PIME](https://github.com/EasyIME/PIME)(LGPL 2.1,使用者自行安裝,本專案不包含其程式碼)
|
||||
- 英文字典:Webster's Second International Dictionary(公有領域,macOS 的 `/usr/share/dict/words`)
|
||||
- AI 選字語言模型:[CKIP GPT2 Tiny Chinese](https://github.com/ckiplab/ckip-transformers)(中研院詞庫小組,GPL-3.0,全文在 `licenses/GPL-3.0.txt`,Mac 版一併放在 app 的 Resources)
|
||||
|
||||
完整的第三方授權聲明(含小麥注音 MIT 授權全文)見 [THIRD_PARTY_LICENSES.txt](THIRD_PARTY_LICENSES.txt),Mac 版放在 `AutoZhuyin.app/Contents/Resources/`,Windows 版放在 zip 根目錄和安裝後的 `autozhuyin` 資料夾。
|
||||
|
||||
@@ -15,9 +15,9 @@
|
||||
<key>CFBundlePackageType</key>
|
||||
<string>APPL</string>
|
||||
<key>CFBundleShortVersionString</key>
|
||||
<string>0.10.1</string>
|
||||
<string>0.11.0</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>18</string>
|
||||
<string>19</string>
|
||||
<key>InputMethodConnectionName</key>
|
||||
<string>com.autozhuyin.inputmethod.AutoZhuyin_Connection</string>
|
||||
<key>InputMethodServerControllerClass</key>
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
import Foundation
|
||||
import AutoZhuyinCore
|
||||
|
||||
/// "AI 選字" settings, model loading and system-load detection for the Mac app.
|
||||
enum AI {
|
||||
static let modeKey = "AIMode"
|
||||
static let autoPauseKey = "AIAutoPause"
|
||||
|
||||
static let reranker: Reranker = {
|
||||
let saved = UserDefaults.standard.string(forKey: modeKey).flatMap(Reranker.Mode.init(rawValue:))
|
||||
let r = Reranker(mode: saved ?? .standard)
|
||||
r.systemState = { SystemMonitor.shared.state() }
|
||||
return r
|
||||
}()
|
||||
|
||||
/// The model file is missing from the app bundle (built without scripts/fetch-lm.sh).
|
||||
private(set) static var modelMissing = false
|
||||
private static var loading = false
|
||||
|
||||
static var autoPause: Bool {
|
||||
get { UserDefaults.standard.object(forKey: autoPauseKey) as? Bool ?? true }
|
||||
set { UserDefaults.standard.set(newValue, forKey: autoPauseKey); SystemMonitor.shared.invalidate() }
|
||||
}
|
||||
|
||||
static func setMode(_ mode: Reranker.Mode) {
|
||||
reranker.mode = mode
|
||||
UserDefaults.standard.set(mode.rawValue, forKey: modeKey)
|
||||
if mode == .off {
|
||||
reranker.model = nil // free the memory
|
||||
} else {
|
||||
loadModelIfNeeded()
|
||||
}
|
||||
}
|
||||
|
||||
/// Loads the model in the background (only when AI is on).
|
||||
static func loadModelIfNeeded() {
|
||||
guard reranker.mode != .off, reranker.model == nil, !loading, !modelMissing else { return }
|
||||
guard let bin = Bundle.main.url(forResource: "lm", withExtension: "bin"),
|
||||
let vocab = Bundle.main.url(forResource: "lm-vocab", withExtension: "txt") else {
|
||||
modelMissing = true
|
||||
return
|
||||
}
|
||||
loading = true
|
||||
DispatchQueue.global(qos: .utility).async {
|
||||
let start = Date()
|
||||
let model = try? GPT2LanguageModel(modelURL: bin, vocabURL: vocab)
|
||||
NSLog("AutoZhuyin: language model %@ in %.0f ms", model == nil ? "failed" : "loaded",
|
||||
Date().timeIntervalSince(start) * 1000)
|
||||
DispatchQueue.main.async {
|
||||
loading = false
|
||||
if model == nil { modelMissing = true }
|
||||
if reranker.mode != .off { reranker.model = model }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static var statusText: String {
|
||||
if modelMissing { return "AI 模型未安裝" }
|
||||
switch reranker.status {
|
||||
case .off: return "AI 選字已關閉"
|
||||
case .loading: return "AI 模型載入中…"
|
||||
case .active:
|
||||
return SystemMonitor.shared.state() == .reduced ? "AI 使用中(省電:低耗電模式)" : "AI 使用中"
|
||||
case .pausedBusy: return "AI 已暫停:系統忙碌或過熱"
|
||||
case .pausedSlow: return "AI 已暫停:運算太慢,一分鐘後再試"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Decides when AI should step back: paused when the Mac is hot or the CPU is busy,
|
||||
/// reduced (fewest candidates) in Low Power Mode. Checked at most every 2 seconds.
|
||||
final class SystemMonitor {
|
||||
static let shared = SystemMonitor()
|
||||
private var cached: (time: Date, state: Reranker.SystemState) = (.distantPast, .normal)
|
||||
/// CPU load (1-minute average per core) above which AI pauses.
|
||||
var busyLoad = 0.9
|
||||
|
||||
func invalidate() { cached.time = .distantPast }
|
||||
|
||||
func state() -> Reranker.SystemState {
|
||||
if Date().timeIntervalSince(cached.time) < 2 { return cached.state }
|
||||
cached = (Date(), compute())
|
||||
return cached.state
|
||||
}
|
||||
|
||||
private func compute() -> Reranker.SystemState {
|
||||
guard AI.autoPause else { return .normal }
|
||||
let info = ProcessInfo.processInfo
|
||||
if info.thermalState == .serious || info.thermalState == .critical { return .paused }
|
||||
var load = [Double](repeating: 0, count: 3)
|
||||
if getloadavg(&load, 3) > 0, load[0] / Double(max(1, info.activeProcessorCount)) > busyLoad {
|
||||
return .paused
|
||||
}
|
||||
if info.isLowPowerModeEnabled { return .reduced }
|
||||
return .normal
|
||||
}
|
||||
}
|
||||
@@ -50,7 +50,11 @@ enum Shared {
|
||||
|
||||
@objc(AutoZhuyinInputController)
|
||||
final class InputController: IMKInputController {
|
||||
private let engine = CompositionEngine(classifier: Shared.classifier)
|
||||
private lazy var engine: CompositionEngine = {
|
||||
let e = CompositionEngine(classifier: Shared.classifier)
|
||||
e.reranker = AI.reranker
|
||||
return e
|
||||
}()
|
||||
private var shiftTap = ShiftTapDetector()
|
||||
|
||||
// MARK: - Key events
|
||||
@@ -182,10 +186,31 @@ final class InputController: IMKInputController {
|
||||
|
||||
override func menu() -> NSMenu! {
|
||||
let menu = NSMenu()
|
||||
let modes: [(Reranker.Mode, String, Selector)] = [
|
||||
(.off, "AI 選字:關閉", #selector(aiOff(_:))),
|
||||
(.saver, "AI 選字:省電", #selector(aiSaver(_:))),
|
||||
(.standard, "AI 選字:標準", #selector(aiStandard(_:))),
|
||||
(.quality, "AI 選字:高品質", #selector(aiQuality(_:))),
|
||||
]
|
||||
for (mode, title, action) in modes {
|
||||
let item = menu.addItem(withTitle: title, action: action, keyEquivalent: "")
|
||||
item.state = AI.reranker.mode == mode ? .on : .off
|
||||
}
|
||||
let pause = menu.addItem(withTitle: "系統忙碌時自動暫停 AI", action: #selector(toggleAutoPause(_:)), keyEquivalent: "")
|
||||
pause.state = AI.autoPause ? .on : .off
|
||||
let status = menu.addItem(withTitle: AI.statusText, action: nil, keyEquivalent: "")
|
||||
status.isEnabled = false
|
||||
menu.addItem(.separator())
|
||||
menu.addItem(withTitle: "清除學習紀錄", action: #selector(clearLearned(_:)), keyEquivalent: "")
|
||||
return menu
|
||||
}
|
||||
|
||||
@objc private func aiOff(_ sender: Any?) { AI.setMode(.off) }
|
||||
@objc private func aiSaver(_ sender: Any?) { AI.setMode(.saver) }
|
||||
@objc private func aiStandard(_ sender: Any?) { AI.setMode(.standard) }
|
||||
@objc private func aiQuality(_ sender: Any?) { AI.setMode(.quality) }
|
||||
@objc private func toggleAutoPause(_ sender: Any?) { AI.autoPause.toggle() }
|
||||
|
||||
@objc private func clearLearned(_ sender: Any?) {
|
||||
Shared.classifier.learned.clear()
|
||||
Shared.classifier.history.clear()
|
||||
|
||||
@@ -22,5 +22,6 @@ else {
|
||||
Shared.server = server
|
||||
Shared.candidates = IMKCandidates(server: server, panelType: kIMKSingleColumnScrollingCandidatePanel)
|
||||
_ = Shared.classifier // load the lexicon now rather than on the first keystroke
|
||||
AI.loadModelIfNeeded() // "AI 選字" language model, in the background
|
||||
|
||||
NSApplication.shared.run()
|
||||
|
||||
@@ -63,6 +63,11 @@ public final class CompositionEngine {
|
||||
public private(set) var pendingBacktick = false
|
||||
static let holdMinScore = -3.5
|
||||
|
||||
/// "AI 選字" (language-model reranking); nil = lexicon only.
|
||||
public var reranker: Reranker?
|
||||
/// The last characters committed, used as context for the reranker.
|
||||
private var recentText = ""
|
||||
|
||||
private var previousMode: InputMode?
|
||||
private var previousCommitTime = Date.distantPast
|
||||
/// How long the previous segment's mode keeps influencing the next one.
|
||||
@@ -186,9 +191,16 @@ public final class CompositionEngine {
|
||||
private func conversion(of seg: Segment, final: Bool) -> (segments: [ConvertedSegment], pending: String) {
|
||||
let (syl, pending) = convertibleSyllables(seg, final: final)
|
||||
let fixedHere = (fixed[seg.range.lowerBound] ?? [:]).filter { $0.key.upperBound <= syl.count }
|
||||
if let reranker, let chosen = reranker.choose(converter, syl, fixed: fixedHere, context: rerankContext) {
|
||||
return (chosen, pending)
|
||||
}
|
||||
return (converter.convert(syl, fixed: fixedHere), pending)
|
||||
}
|
||||
|
||||
private var rerankContext: String {
|
||||
now().timeIntervalSince(previousCommitTime) < contextWindow ? recentText : ""
|
||||
}
|
||||
|
||||
// MARK: - Cursor
|
||||
|
||||
/// Absolute key index after each syllable of a Zhuyin segment.
|
||||
@@ -542,6 +554,7 @@ public final class CompositionEngine {
|
||||
let segs = given ?? segments
|
||||
let text = segs.map { render($0, final: true) }.joined()
|
||||
rememberPicks(segs)
|
||||
recentText = String((rerankContext + text).suffix(20))
|
||||
if let forced = forcedMode {
|
||||
classifier.learned.record(String(keys), as: forced)
|
||||
}
|
||||
|
||||
@@ -61,6 +61,32 @@ public struct Converter {
|
||||
return segments.reversed()
|
||||
}
|
||||
|
||||
/// The `k` best paths (log10 score, segments), best first — for language-model reranking.
|
||||
public func kBest(_ syllables: [Syllable], fixed: [Range<Int>: String] = [:], k: Int) -> [(score: Double, segments: [ConvertedSegment])] {
|
||||
let n = syllables.count
|
||||
guard n > 0, k > 0 else { return [] }
|
||||
let maxLen = max(1, lexicon.maxPhraseLength)
|
||||
typealias Path = (score: Double, segments: [ConvertedSegment])
|
||||
var beams: [[Path]] = Array(repeating: [], count: n + 1)
|
||||
beams[0] = [(0, [])]
|
||||
for j in 1...n {
|
||||
var cand: [Path] = []
|
||||
for len in 1...min(maxLen, j) {
|
||||
let i = j - len
|
||||
guard !beams[i].isEmpty else { continue }
|
||||
let range = i..<j
|
||||
for (text, score) in options(for: range, syllables: syllables, fixed: fixed) {
|
||||
for path in beams[i] {
|
||||
cand.append((path.score + score, path.segments + [ConvertedSegment(range: range, text: text)]))
|
||||
}
|
||||
}
|
||||
}
|
||||
cand.sort { $0.score > $1.score }
|
||||
beams[j] = Array(cand.prefix(k))
|
||||
}
|
||||
return beams[n]
|
||||
}
|
||||
|
||||
private func options(for range: Range<Int>, syllables: [Syllable], fixed: [Range<Int>: String]) -> [(String, Double)] {
|
||||
if let text = fixed[range] { return [(text, 0)] }
|
||||
// A span may not partially overlap or swallow a fixed span.
|
||||
|
||||
@@ -0,0 +1,242 @@
|
||||
import Accelerate
|
||||
import Foundation
|
||||
|
||||
/// Scores Chinese text: natural-log probability of `text` following `context`.
|
||||
public protocol LanguageModel: AnyObject {
|
||||
func logProb(_ text: String, context: String) -> Double
|
||||
}
|
||||
|
||||
/// GPT-2 forward pass (CPU, Accelerate) for the character-level CKIP gpt2-tiny-chinese
|
||||
/// model (GPL-3.0), stored by training/lm/export_lm.py as float16 ("AZLM" format).
|
||||
/// Weights are expanded to float32 in memory (~46 MB for gpt2-tiny).
|
||||
public final class GPT2LanguageModel: LanguageModel {
|
||||
public enum LoadError: Error { case badFormat(String) }
|
||||
|
||||
let nLayer: Int, nHead: Int, d: Int, vocabSize: Int
|
||||
let eps: Float
|
||||
private var weights: [Float] = []
|
||||
private var offsets: [String: Int] = [:]
|
||||
private var vocab: [Character: Int32] = [:]
|
||||
static let cls: Int32 = 101, unk: Int32 = 100
|
||||
/// Longest input scored (context + text); longer contexts are cut from the left.
|
||||
public var maxTokens = 64
|
||||
|
||||
public init(modelURL: URL, vocabURL: URL) throws {
|
||||
let data = try Data(contentsOf: modelURL, options: .mappedIfSafe)
|
||||
guard data.count > 8, data.prefix(4) == Data("AZLM".utf8) else { throw LoadError.badFormat("magic") }
|
||||
let headerLength = Int(data[4]) | Int(data[5]) << 8 | Int(data[6]) << 16 | Int(data[7]) << 24
|
||||
guard let header = try JSONSerialization.jsonObject(with: data.subdata(in: 8..<(8 + headerLength))) as? [String: Any],
|
||||
let nLayer = header["n_layer"] as? Int, let nHead = header["n_head"] as? Int,
|
||||
let d = header["n_embd"] as? Int, let vocabSize = header["vocab"] as? Int,
|
||||
let eps = header["eps"] as? Double, let tensors = header["tensors"] as? [[Any]]
|
||||
else { throw LoadError.badFormat("header") }
|
||||
self.nLayer = nLayer; self.nHead = nHead; self.d = d; self.vocabSize = vocabSize; self.eps = Float(eps)
|
||||
for t in tensors {
|
||||
guard let name = t[0] as? String, let offset = t[2] as? Int else { throw LoadError.badFormat("tensor") }
|
||||
offsets[name] = offset
|
||||
}
|
||||
let start = 8 + headerLength
|
||||
let count = (data.count - start) / 2
|
||||
weights = [Float](repeating: 0, count: count)
|
||||
try data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) in
|
||||
guard let base = raw.baseAddress?.advanced(by: start) else { throw LoadError.badFormat("data") }
|
||||
try weights.withUnsafeMutableBufferPointer { out in
|
||||
var src = vImage_Buffer(data: UnsafeMutableRawPointer(mutating: base), height: 1,
|
||||
width: vImagePixelCount(count), rowBytes: count * 2)
|
||||
var dst = vImage_Buffer(data: out.baseAddress, height: 1,
|
||||
width: vImagePixelCount(count), rowBytes: count * 4)
|
||||
guard vImageConvert_Planar16FtoPlanarF(&src, &dst, 0) == kvImageNoError else {
|
||||
throw LoadError.badFormat("float16")
|
||||
}
|
||||
}
|
||||
}
|
||||
var index: Int32 = 0
|
||||
for line in try String(contentsOf: vocabURL, encoding: .utf8).split(separator: "\n", omittingEmptySubsequences: false) {
|
||||
if line.count == 1, let c = line.first, vocab[c] == nil { vocab[c] = index }
|
||||
index += 1
|
||||
}
|
||||
for name in ["wte.weight", "wpe.weight", "ln_f.weight"] where offsets[name] == nil {
|
||||
throw LoadError.badFormat("missing \(name)")
|
||||
}
|
||||
}
|
||||
|
||||
func tokens(_ text: String) -> [Int32] { text.map { vocab[$0] ?? GPT2LanguageModel.unk } }
|
||||
|
||||
public func logProb(_ text: String, context: String) -> Double {
|
||||
let target = tokens(text)
|
||||
guard !target.isEmpty else { return 0 }
|
||||
var ctx = tokens(context)
|
||||
if ctx.count + target.count + 1 > maxTokens { ctx = Array(ctx.suffix(max(0, maxTokens - target.count - 1))) }
|
||||
let input = [GPT2LanguageModel.cls] + ctx + target.dropLast()
|
||||
let h = hidden(input) // [T, d]
|
||||
return weights.withUnsafeBufferPointer { w in
|
||||
logSoftmaxScore(h, rows: input.count, from: ctx.count, targets: target, w: w)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Forward pass
|
||||
|
||||
private func w(_ name: String) -> Int { offsets[name]! }
|
||||
|
||||
/// Final hidden states [T, d] (row-major).
|
||||
func hidden(_ ids: [Int32]) -> [Float] {
|
||||
let T = ids.count, d = self.d
|
||||
var x = [Float](repeating: 0, count: T * d)
|
||||
weights.withUnsafeBufferPointer { W in
|
||||
let wte = w("wte.weight"), wpe = w("wpe.weight")
|
||||
for t in 0..<T {
|
||||
vDSP_vadd(W.baseAddress! + wte + Int(ids[t]) * d, 1, W.baseAddress! + wpe + t * d, 1,
|
||||
&x[t * d], 1, vDSP_Length(d))
|
||||
}
|
||||
for l in 0..<nLayer {
|
||||
let p = "h.\(l)."
|
||||
var h = layerNorm(x, rows: T, p + "ln_1", W)
|
||||
let qkv = linear(h, rows: T, inDim: d, outDim: 3 * d, p + "attn.c_attn", W)
|
||||
let a = attention(qkv, T: T)
|
||||
var proj = linear(a, rows: T, inDim: d, outDim: d, p + "attn.c_proj", W)
|
||||
GPT2LanguageModel.add(&x, proj)
|
||||
h = layerNorm(x, rows: T, p + "ln_2", W)
|
||||
var f = linear(h, rows: T, inDim: d, outDim: 4 * d, p + "mlp.c_fc", W)
|
||||
geluNew(&f)
|
||||
proj = linear(f, rows: T, inDim: 4 * d, outDim: d, p + "mlp.c_proj", W)
|
||||
GPT2LanguageModel.add(&x, proj)
|
||||
}
|
||||
x = layerNorm(x, rows: T, "ln_f", W)
|
||||
}
|
||||
return x
|
||||
}
|
||||
|
||||
private func layerNorm(_ x: [Float], rows: Int, _ name: String, _ W: UnsafeBufferPointer<Float>) -> [Float] {
|
||||
let d = self.d
|
||||
var out = [Float](repeating: 0, count: x.count)
|
||||
let g = W.baseAddress! + w(name + ".weight"), b = W.baseAddress! + w(name + ".bias")
|
||||
for r in 0..<rows {
|
||||
var mean: Float = 0, sd: Float = 0
|
||||
x.withUnsafeBufferPointer { xp in
|
||||
out.withUnsafeMutableBufferPointer { op in
|
||||
let src = xp.baseAddress! + r * d, dst = op.baseAddress! + r * d
|
||||
vDSP_normalize(src, 1, dst, 1, &mean, &sd, vDSP_Length(d))
|
||||
// vDSP_normalize divides by the population std without eps; redo with eps.
|
||||
let scale = sd / (sd * sd + eps).squareRoot()
|
||||
var s = scale
|
||||
vDSP_vsmul(dst, 1, &s, dst, 1, vDSP_Length(d))
|
||||
vDSP_vma(dst, 1, g, 1, b, 1, dst, 1, vDSP_Length(d))
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/// x [rows, inDim] · W [inDim, outDim] + bias.
|
||||
private func linear(_ x: [Float], rows: Int, inDim: Int, outDim: Int, _ name: String,
|
||||
_ W: UnsafeBufferPointer<Float>) -> [Float] {
|
||||
var out = [Float](repeating: 0, count: rows * outDim)
|
||||
vDSP_mmul(x, 1, W.baseAddress! + w(name + ".weight"), 1, &out, 1,
|
||||
vDSP_Length(rows), vDSP_Length(outDim), vDSP_Length(inDim))
|
||||
let bias = W.baseAddress! + w(name + ".bias")
|
||||
out.withUnsafeMutableBufferPointer { op in
|
||||
for r in 0..<rows {
|
||||
vDSP_vadd(op.baseAddress! + r * outDim, 1, bias, 1, op.baseAddress! + r * outDim, 1, vDSP_Length(outDim))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/// Causal multi-head self-attention. qkv is [T, 3d] (q | k | v).
|
||||
private func attention(_ qkv: [Float], T: Int) -> [Float] {
|
||||
let d = self.d, hd = d / nHead
|
||||
let scale = 1 / Float(hd).squareRoot()
|
||||
var out = [Float](repeating: 0, count: T * d)
|
||||
var scores = [Float](repeating: 0, count: T)
|
||||
qkv.withUnsafeBufferPointer { q in
|
||||
out.withUnsafeMutableBufferPointer { o in
|
||||
for head in 0..<nHead {
|
||||
for i in 0..<T {
|
||||
let qi = q.baseAddress! + i * 3 * d + head * hd
|
||||
var maxScore = -Float.infinity
|
||||
for j in 0...i {
|
||||
var s: Float = 0
|
||||
vDSP_dotpr(qi, 1, q.baseAddress! + j * 3 * d + d + head * hd, 1, &s, vDSP_Length(hd))
|
||||
scores[j] = s * scale
|
||||
maxScore = max(maxScore, scores[j])
|
||||
}
|
||||
var sum: Float = 0
|
||||
for j in 0...i { scores[j] = expf(scores[j] - maxScore); sum += scores[j] }
|
||||
let dst = o.baseAddress! + i * d + head * hd
|
||||
for j in 0...i {
|
||||
var a = scores[j] / sum
|
||||
vDSP_vsma(q.baseAddress! + j * 3 * d + 2 * d + head * hd, 1, &a, dst, 1, dst, 1, vDSP_Length(hd))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/// x += y (same length).
|
||||
private static func add(_ x: inout [Float], _ y: [Float]) {
|
||||
x.withUnsafeMutableBufferPointer { xp in
|
||||
vDSP_vadd(xp.baseAddress!, 1, y, 1, xp.baseAddress!, 1, vDSP_Length(xp.count))
|
||||
}
|
||||
}
|
||||
|
||||
/// gelu_new: 0.5·x·(1 + tanh(√(2/π)·(x + 0.044715·x³))).
|
||||
private func geluNew(_ x: inout [Float]) {
|
||||
let n = x.count
|
||||
var inner = [Float](repeating: 0, count: n)
|
||||
x.withUnsafeMutableBufferPointer { xb in
|
||||
inner.withUnsafeMutableBufferPointer { ib in
|
||||
let xp = xb.baseAddress!, ip = ib.baseAddress!, len = vDSP_Length(n)
|
||||
var c3: Float = 0.044715, one: Float = 1, k = (2 / Float.pi).squareRoot(), half: Float = 0.5
|
||||
vDSP_vsq(xp, 1, ip, 1, len) // x²
|
||||
vDSP_vsmsa(ip, 1, &c3, &one, ip, 1, len) // 1 + 0.044715·x²
|
||||
vDSP_vmul(ip, 1, xp, 1, ip, 1, len) // x + 0.044715·x³
|
||||
vDSP_vsmul(ip, 1, &k, ip, 1, len)
|
||||
var count = Int32(n)
|
||||
vvtanhf(ip, ip, &count)
|
||||
vDSP_vsadd(ip, 1, &one, ip, 1, len) // 1 + tanh(…)
|
||||
vDSP_vmul(ip, 1, xp, 1, xp, 1, len)
|
||||
vDSP_vsmul(xp, 1, &half, xp, 1, len)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Sum over target positions of log softmax(h · wteᵀ)[target]. Row `from + i` predicts target i.
|
||||
private func logSoftmaxScore(_ h: [Float], rows T: Int, from start: Int, targets: [Int32],
|
||||
w W: UnsafeBufferPointer<Float>) -> Double {
|
||||
let d = self.d, V = vocabSize
|
||||
let rowsUsed = T - start
|
||||
// hᵀ for the rows we need: [d, rowsUsed]
|
||||
var hT = [Float](repeating: 0, count: d * rowsUsed)
|
||||
h.withUnsafeBufferPointer { hp in
|
||||
vDSP_mtrans(hp.baseAddress! + start * d, 1, &hT, 1, vDSP_Length(d), vDSP_Length(rowsUsed))
|
||||
}
|
||||
// logitsᵀ [V, rowsUsed] = wte [V, d] · hᵀ [d, rowsUsed]
|
||||
var logitsT = [Float](repeating: 0, count: V * rowsUsed)
|
||||
vDSP_mmul(W.baseAddress! + w("wte.weight"), 1, hT, 1, &logitsT, 1,
|
||||
vDSP_Length(V), vDSP_Length(rowsUsed), vDSP_Length(d))
|
||||
var logits = [Float](repeating: 0, count: V * rowsUsed) // [rowsUsed, V]
|
||||
vDSP_mtrans(logitsT, 1, &logits, 1, vDSP_Length(rowsUsed), vDSP_Length(V))
|
||||
var total = 0.0
|
||||
var expBuf = [Float](repeating: 0, count: V)
|
||||
logits.withUnsafeMutableBufferPointer { lp in
|
||||
expBuf.withUnsafeMutableBufferPointer { eb in
|
||||
let e = eb.baseAddress!
|
||||
for i in 0..<rowsUsed {
|
||||
let row = lp.baseAddress! + i * V
|
||||
var m: Float = 0
|
||||
vDSP_maxv(row, 1, &m, vDSP_Length(V))
|
||||
var negM = -m
|
||||
vDSP_vsadd(row, 1, &negM, e, 1, vDSP_Length(V))
|
||||
var n = Int32(V)
|
||||
vvexpf(e, e, &n)
|
||||
var sum: Float = 0
|
||||
vDSP_sve(e, 1, &sum, vDSP_Length(V))
|
||||
total += Double(row[Int(targets[i])] - m) - Double(logf(sum))
|
||||
}
|
||||
}
|
||||
}
|
||||
return total
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
import Foundation
|
||||
|
||||
/// "AI 選字": reranks the converter's best paths with a language model.
|
||||
///
|
||||
/// score = lexicon score (log10 → ln) + λ · LM log-probability
|
||||
///
|
||||
/// It never blocks typing for long: each rerank has a time budget, and a rerank that runs
|
||||
/// over it is dropped (the plain lexicon result is used). Three overruns in a row pause
|
||||
/// AI for a minute. `systemState` lets the app pause or reduce it when the machine is
|
||||
/// busy, hot or saving power.
|
||||
public final class Reranker {
|
||||
public enum Mode: String, CaseIterable {
|
||||
case off, saver, standard, quality
|
||||
|
||||
/// How many paths are reranked, and the time budget for one rerank.
|
||||
var k: Int { [.saver: 3, .standard: 10, .quality: 30][self] ?? 0 }
|
||||
var budget: TimeInterval { [.saver: 0.015, .standard: 0.04, .quality: 0.12][self] ?? 0 }
|
||||
}
|
||||
|
||||
/// What the system allows right now (decided by the app).
|
||||
public enum SystemState { case normal, reduced, paused }
|
||||
|
||||
public enum Status: Equatable {
|
||||
case off, loading, active, pausedBusy, pausedSlow
|
||||
}
|
||||
|
||||
public var mode: Mode { didSet { cache.removeAll() } }
|
||||
/// nil while loading (or if the model file is missing).
|
||||
public var model: LanguageModel? { didSet { cache.removeAll() } }
|
||||
public var lambda = 0.6
|
||||
public var systemState: () -> SystemState = { .normal }
|
||||
public var now: () -> Date = Date.init
|
||||
public private(set) var status: Status = .off
|
||||
|
||||
private var overruns = 0
|
||||
private var pausedUntil = Date.distantPast
|
||||
private var cache: [String: [ConvertedSegment]] = [:]
|
||||
static let cacheLimit = 512
|
||||
static let pauseAfterOverruns = 3
|
||||
static let slowPause: TimeInterval = 60
|
||||
|
||||
public init(mode: Mode = .standard, model: LanguageModel? = nil) {
|
||||
self.mode = mode
|
||||
self.model = model
|
||||
}
|
||||
|
||||
/// The mode actually used now (after the system state), or nil when AI is not used.
|
||||
var effectiveMode: Mode? {
|
||||
guard mode != .off else { status = .off; return nil }
|
||||
guard model != nil else { status = .loading; return nil }
|
||||
if now() < pausedUntil { status = .pausedSlow; return nil }
|
||||
switch systemState() {
|
||||
case .paused: status = .pausedBusy; return nil
|
||||
case .reduced: status = .active; return .saver
|
||||
case .normal: status = .active; return mode
|
||||
}
|
||||
}
|
||||
|
||||
/// The reranked conversion, or nil to use the plain lexicon result.
|
||||
func choose(_ converter: Converter, _ syllables: [Syllable], fixed: [Range<Int>: String],
|
||||
context: String) -> [ConvertedSegment]? {
|
||||
guard let mode = effectiveMode, let model, !syllables.isEmpty else { return nil }
|
||||
let key = "\(mode.rawValue)|\(context)|\(syllables.map(\.reading).joined(separator: "-"))|"
|
||||
+ fixed.sorted { $0.key.lowerBound < $1.key.lowerBound }.map { "\($0.key)=\($0.value)" }.joined(separator: ",")
|
||||
if let hit = cache[key] { return hit }
|
||||
|
||||
let paths = converter.kBest(syllables, fixed: fixed, k: mode.k)
|
||||
guard paths.count > 1 else { return paths.first?.segments }
|
||||
let start = now()
|
||||
var best: (score: Double, segments: [ConvertedSegment])?
|
||||
for path in paths {
|
||||
let text = path.segments.map(\.text).joined()
|
||||
let score = path.score * log(10) + lambda * model.logProb(text, context: context)
|
||||
if best == nil || score > best!.score { best = (score, path.segments) }
|
||||
if now().timeIntervalSince(start) > mode.budget {
|
||||
overruns += 1
|
||||
if overruns >= Reranker.pauseAfterOverruns {
|
||||
pausedUntil = now().addingTimeInterval(Reranker.slowPause)
|
||||
overruns = 0
|
||||
status = .pausedSlow
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
overruns = 0
|
||||
if cache.count >= Reranker.cacheLimit { cache.removeAll() }
|
||||
cache[key] = best?.segments
|
||||
return best?.segments
|
||||
}
|
||||
}
|
||||
@@ -48,3 +48,17 @@ https://github.com/EasyIME/PIME — GNU LGPL 2.1
|
||||
|
||||
The Windows version runs as a module inside PIME, which users install
|
||||
separately. No PIME code is included in this software.
|
||||
|
||||
|
||||
4. CKIP GPT2 Tiny Chinese — AI 選字語言模型 (language model, macOS)
|
||||
-------------------------------------------------------------------
|
||||
https://huggingface.co/ckiplab/gpt2-tiny-chinese
|
||||
https://github.com/ckiplab/ckip-transformers
|
||||
Author: Mu Yang at CKIP, Academia Sinica (中央研究院 詞庫小組)
|
||||
License: GNU General Public License v3.0 (full text: GPL-3.0.txt)
|
||||
|
||||
lm.bin is the model's weights converted to float16 by training/lm/export_lm.py;
|
||||
lm-vocab.txt is its vocabulary (from bert-base-chinese). They are used unmodified
|
||||
otherwise, to rerank conversion candidates ("AI 選字"). The source code of this input
|
||||
method, including the conversion script and the code that runs the model, is
|
||||
available from the project repository.
|
||||
|
||||
@@ -641,6 +641,126 @@ final class UserHistoryTests: XCTestCase {
|
||||
}
|
||||
}
|
||||
|
||||
/// Needs build/lm/lm.bin (scripts/fetch-lm.sh); skipped otherwise.
|
||||
final class GPT2LanguageModelTests: XCTestCase {
|
||||
static let dir = resources.deletingLastPathComponent().appendingPathComponent("build/lm")
|
||||
|
||||
func model() throws -> GPT2LanguageModel {
|
||||
let bin = GPT2LanguageModelTests.dir.appendingPathComponent("lm.bin")
|
||||
try XCTSkipUnless(FileManager.default.fileExists(atPath: bin.path), "run scripts/fetch-lm.sh")
|
||||
return try GPT2LanguageModel(modelURL: bin, vocabURL: GPT2LanguageModelTests.dir.appendingPathComponent("lm-vocab.txt"))
|
||||
}
|
||||
|
||||
/// Same numbers as the numpy reference (training/lm/gpt2.py, float32 weights).
|
||||
func testMatchesReference() throws {
|
||||
let m = try model()
|
||||
let cases: [(String, String, Double)] = [("他在家", "", -14.329), ("今天天氣很好", "", -30.08),
|
||||
("今添天氣很好", "", -44.392), ("家", "他在", -5.514),
|
||||
("天氣很好", "今天", -23.883)]
|
||||
for (text, context, want) in cases {
|
||||
XCTAssertEqual(m.logProb(text, context: context), want, accuracy: 0.05, text)
|
||||
}
|
||||
}
|
||||
|
||||
func testSpeed() throws {
|
||||
let m = try model()
|
||||
let start = Date()
|
||||
for _ in 0..<20 { _ = m.logProb("我們明天早上八點在車站見面", context: "好啊") }
|
||||
let ms = Date().timeIntervalSince(start) / 20 * 1000
|
||||
print("GPT2 logProb: \(String(format: "%.2f", ms)) ms")
|
||||
XCTAssertLessThan(ms, 20)
|
||||
}
|
||||
}
|
||||
|
||||
/// A language model that likes one string and can pretend to be slow.
|
||||
private final class FakeLanguageModel: LanguageModel {
|
||||
var liked: String
|
||||
var onCall: () -> Void = {}
|
||||
init(liked: String) { self.liked = liked }
|
||||
func logProb(_ text: String, context: String) -> Double {
|
||||
onCall()
|
||||
return text.contains(liked) ? 0 : -50
|
||||
}
|
||||
}
|
||||
|
||||
final class RerankerTests: XCTestCase {
|
||||
func engine(_ reranker: Reranker?) -> CompositionEngine {
|
||||
let e = CompositionEngine(classifier: Classifier(lexicon: sharedLexicon, english: sharedEnglish, weights: sharedWeights))
|
||||
e.reranker = reranker
|
||||
return e
|
||||
}
|
||||
|
||||
func type(_ s: String, into e: CompositionEngine) -> String {
|
||||
var out = ""
|
||||
for c in s {
|
||||
let r = e.handle(c == "\n" ? .enter : .character(c))
|
||||
if let t = r.commit { out += t } else if !r.handled { out.append(c) }
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func testRerankPicksWhatTheModelLikes() {
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(nil)), "天")
|
||||
let r = Reranker(mode: .standard, model: FakeLanguageModel(liked: "添"))
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(r)), "添")
|
||||
XCTAssertEqual(r.status, .active)
|
||||
}
|
||||
|
||||
func testOffAndLoadingUseTheLexicon() {
|
||||
let off = Reranker(mode: .off, model: FakeLanguageModel(liked: "添"))
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(off)), "天")
|
||||
XCTAssertEqual(off.status, .off)
|
||||
let loading = Reranker(mode: .standard, model: nil)
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(loading)), "天")
|
||||
XCTAssertEqual(loading.status, .loading)
|
||||
}
|
||||
|
||||
func testPausedWhenSystemBusy() {
|
||||
let r = Reranker(mode: .standard, model: FakeLanguageModel(liked: "添"))
|
||||
r.systemState = { .paused }
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(r)), "天")
|
||||
XCTAssertEqual(r.status, .pausedBusy)
|
||||
}
|
||||
|
||||
func testReducedUsesFewerPaths() {
|
||||
let lm = FakeLanguageModel(liked: "添")
|
||||
var calls = 0
|
||||
lm.onCall = { calls += 1 }
|
||||
let r = Reranker(mode: .quality, model: lm)
|
||||
r.systemState = { .reduced }
|
||||
_ = type("wu0 \n", into: engine(r))
|
||||
XCTAssertLessThanOrEqual(calls, 3)
|
||||
}
|
||||
|
||||
func testSlowModelIsDroppedThenPaused() {
|
||||
var clock = Date(timeIntervalSince1970: 0)
|
||||
let lm = FakeLanguageModel(liked: "添")
|
||||
lm.onCall = { clock.addTimeInterval(1) } // every call takes a "second"
|
||||
let r = Reranker(mode: .standard, model: lm)
|
||||
r.now = { clock }
|
||||
for _ in 0..<3 { XCTAssertEqual(type("wu0 \n", into: engine(r)), "天") }
|
||||
XCTAssertEqual(r.status, .pausedSlow)
|
||||
lm.onCall = {}
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(r)), "天") // still paused
|
||||
clock.addTimeInterval(61)
|
||||
XCTAssertEqual(type("wu0 \n", into: engine(r)), "添") // back after a minute
|
||||
}
|
||||
|
||||
/// With the real model (build/lm, skipped if missing): sentences the lexicon alone gets wrong.
|
||||
func testRealModelFixesHomophones() throws {
|
||||
let dir = GPT2LanguageModelTests.dir
|
||||
let bin = dir.appendingPathComponent("lm.bin")
|
||||
try XCTSkipUnless(FileManager.default.fileExists(atPath: bin.path), "run scripts/fetch-lm.sh")
|
||||
let lm = try GPT2LanguageModel(modelURL: bin, vocabURL: dir.appendingPathComponent("lm-vocab.txt"))
|
||||
let cases = ["w8 ql32k71u3ji3dj94cp32ji ": "他跑得比我快很多",
|
||||
"fu/3su3y94gji u h4cl3a87": "請你再說一次好嗎"]
|
||||
for (keys, want) in cases {
|
||||
XCTAssertNotEqual(type(keys + "\n", into: engine(nil)), want)
|
||||
XCTAssertEqual(type(keys + "\n", into: engine(Reranker(mode: .standard, model: lm))), want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
final class StartupTimingTests: XCTestCase {
|
||||
func testStartupTiming() throws {
|
||||
var t = Date()
|
||||
|
||||
@@ -0,0 +1,674 @@
|
||||
GNU GENERAL PUBLIC LICENSE
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
Preamble
|
||||
|
||||
The GNU General Public License is a free, copyleft license for
|
||||
software and other kinds of works.
|
||||
|
||||
The licenses for most software and other practical works are designed
|
||||
to take away your freedom to share and change the works. By contrast,
|
||||
the GNU General Public License is intended to guarantee your freedom to
|
||||
share and change all versions of a program--to make sure it remains free
|
||||
software for all its users. We, the Free Software Foundation, use the
|
||||
GNU General Public License for most of our software; it applies also to
|
||||
any other work released this way by its authors. You can apply it to
|
||||
your programs, too.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
them if you wish), that you receive source code or can get it if you
|
||||
want it, that you can change the software or use pieces of it in new
|
||||
free programs, and that you know you can do these things.
|
||||
|
||||
To protect your rights, we need to prevent others from denying you
|
||||
these rights or asking you to surrender the rights. Therefore, you have
|
||||
certain responsibilities if you distribute copies of the software, or if
|
||||
you modify it: responsibilities to respect the freedom of others.
|
||||
|
||||
For example, if you distribute copies of such a program, whether
|
||||
gratis or for a fee, you must pass on to the recipients the same
|
||||
freedoms that you received. You must make sure that they, too, receive
|
||||
or can get the source code. And you must show them these terms so they
|
||||
know their rights.
|
||||
|
||||
Developers that use the GNU GPL protect your rights with two steps:
|
||||
(1) assert copyright on the software, and (2) offer you this License
|
||||
giving you legal permission to copy, distribute and/or modify it.
|
||||
|
||||
For the developers' and authors' protection, the GPL clearly explains
|
||||
that there is no warranty for this free software. For both users' and
|
||||
authors' sake, the GPL requires that modified versions be marked as
|
||||
changed, so that their problems will not be attributed erroneously to
|
||||
authors of previous versions.
|
||||
|
||||
Some devices are designed to deny users access to install or run
|
||||
modified versions of the software inside them, although the manufacturer
|
||||
can do so. This is fundamentally incompatible with the aim of
|
||||
protecting users' freedom to change the software. The systematic
|
||||
pattern of such abuse occurs in the area of products for individuals to
|
||||
use, which is precisely where it is most unacceptable. Therefore, we
|
||||
have designed this version of the GPL to prohibit the practice for those
|
||||
products. If such problems arise substantially in other domains, we
|
||||
stand ready to extend this provision to those domains in future versions
|
||||
of the GPL, as needed to protect the freedom of users.
|
||||
|
||||
Finally, every program is threatened constantly by software patents.
|
||||
States should not allow patents to restrict development and use of
|
||||
software on general-purpose computers, but in those that do, we wish to
|
||||
avoid the special danger that patents applied to a free program could
|
||||
make it effectively proprietary. To prevent this, the GPL assures that
|
||||
patents cannot be used to render the program non-free.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||
works, such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this
|
||||
License. Each licensee is addressed as "you". "Licensees" and
|
||||
"recipients" may be individuals or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work
|
||||
in a fashion requiring copyright permission, other than the making of an
|
||||
exact copy. The resulting work is called a "modified version" of the
|
||||
earlier work or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based
|
||||
on the Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without
|
||||
permission, would make you directly or secondarily liable for
|
||||
infringement under applicable copyright law, except executing it on a
|
||||
computer or modifying a private copy. Propagation includes copying,
|
||||
distribution (with or without modification), making available to the
|
||||
public, and in some countries other activities as well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other
|
||||
parties to make or receive copies. Mere interaction with a user through
|
||||
a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices"
|
||||
to the extent that it includes a convenient and prominently visible
|
||||
feature that (1) displays an appropriate copyright notice, and (2)
|
||||
tells the user that there is no warranty for the work (except to the
|
||||
extent that warranties are provided), that licensees may convey the
|
||||
work under this License, and how to view a copy of this License. If
|
||||
the interface presents a list of user commands or options, such as a
|
||||
menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
|
||||
The "source code" for a work means the preferred form of the work
|
||||
for making modifications to it. "Object code" means any non-source
|
||||
form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official
|
||||
standard defined by a recognized standards body, or, in the case of
|
||||
interfaces specified for a particular programming language, one that
|
||||
is widely used among developers working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other
|
||||
than the work as a whole, that (a) is included in the normal form of
|
||||
packaging a Major Component, but which is not part of that Major
|
||||
Component, and (b) serves only to enable use of the work with that
|
||||
Major Component, or to implement a Standard Interface for which an
|
||||
implementation is available to the public in source code form. A
|
||||
"Major Component", in this context, means a major essential component
|
||||
(kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to
|
||||
produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all
|
||||
the source code needed to generate, install, and (for an executable
|
||||
work) run the object code and to modify the work, including scripts to
|
||||
control those activities. However, it does not include the work's
|
||||
System Libraries, or general-purpose tools or generally available free
|
||||
programs which are used unmodified in performing those activities but
|
||||
which are not part of the work. For example, Corresponding Source
|
||||
includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically
|
||||
linked subprograms that the work is specifically designed to require,
|
||||
such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users
|
||||
can regenerate automatically from other parts of the Corresponding
|
||||
Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that
|
||||
same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of
|
||||
copyright on the Program, and are irrevocable provided the stated
|
||||
conditions are met. This License explicitly affirms your unlimited
|
||||
permission to run the unmodified Program. The output from running a
|
||||
covered work is covered by this License only if the output, given its
|
||||
content, constitutes a covered work. This License acknowledges your
|
||||
rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not
|
||||
convey, without conditions so long as your license otherwise remains
|
||||
in force. You may convey covered works to others for the sole purpose
|
||||
of having them make modifications exclusively for you, or provide you
|
||||
with facilities for running those works, provided that you comply with
|
||||
the terms of this License in conveying all material for which you do
|
||||
not control copyright. Those thus making or running the covered works
|
||||
for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of
|
||||
your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under
|
||||
the conditions stated below. Sublicensing is not allowed; section 10
|
||||
makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological
|
||||
measure under any applicable law fulfilling obligations under article
|
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||
similar laws prohibiting or restricting circumvention of such
|
||||
measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid
|
||||
circumvention of technological measures to the extent such circumvention
|
||||
is effected by exercising rights under this License with respect to
|
||||
the covered work, and you disclaim any intention to limit operation or
|
||||
modification of the work as a means of enforcing, against the work's
|
||||
users, your or third parties' legal rights to forbid circumvention of
|
||||
technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you
|
||||
receive it, in any medium, provided that you conspicuously and
|
||||
appropriately publish on each copy an appropriate copyright notice;
|
||||
keep intact all notices stating that this License and any
|
||||
non-permissive terms added in accord with section 7 apply to the code;
|
||||
keep intact all notices of the absence of any warranty; and give all
|
||||
recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey,
|
||||
and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to
|
||||
produce it from the Program, in the form of source code under the
|
||||
terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified
|
||||
it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is
|
||||
released under this License and any conditions added under section
|
||||
7. This requirement modifies the requirement in section 4 to
|
||||
"keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this
|
||||
License to anyone who comes into possession of a copy. This
|
||||
License will therefore apply, along with any applicable section 7
|
||||
additional terms, to the whole of the work, and all its parts,
|
||||
regardless of how they are packaged. This License gives no
|
||||
permission to license the work in any other way, but it does not
|
||||
invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display
|
||||
Appropriate Legal Notices; however, if the Program has interactive
|
||||
interfaces that do not display Appropriate Legal Notices, your
|
||||
work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent
|
||||
works, which are not by their nature extensions of the covered work,
|
||||
and which are not combined with it such as to form a larger program,
|
||||
in or on a volume of a storage or distribution medium, is called an
|
||||
"aggregate" if the compilation and its resulting copyright are not
|
||||
used to limit the access or legal rights of the compilation's users
|
||||
beyond what the individual works permit. Inclusion of a covered work
|
||||
in an aggregate does not cause this License to apply to the other
|
||||
parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms
|
||||
of sections 4 and 5, provided that you also convey the
|
||||
machine-readable Corresponding Source under the terms of this License,
|
||||
in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by the
|
||||
Corresponding Source fixed on a durable physical medium
|
||||
customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by a
|
||||
written offer, valid for at least three years and valid for as
|
||||
long as you offer spare parts or customer support for that product
|
||||
model, to give anyone who possesses the object code either (1) a
|
||||
copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical
|
||||
medium customarily used for software interchange, for a price no
|
||||
more than your reasonable cost of physically performing this
|
||||
conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the
|
||||
written offer to provide the Corresponding Source. This
|
||||
alternative is allowed only occasionally and noncommercially, and
|
||||
only if you received the object code with such an offer, in accord
|
||||
with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated
|
||||
place (gratis or for a charge), and offer equivalent access to the
|
||||
Corresponding Source in the same way through the same place at no
|
||||
further charge. You need not require recipients to copy the
|
||||
Corresponding Source along with the object code. If the place to
|
||||
copy the object code is a network server, the Corresponding Source
|
||||
may be on a different server (operated by you or a third party)
|
||||
that supports equivalent copying facilities, provided you maintain
|
||||
clear directions next to the object code saying where to find the
|
||||
Corresponding Source. Regardless of what server hosts the
|
||||
Corresponding Source, you remain obligated to ensure that it is
|
||||
available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided
|
||||
you inform other peers where the object code and Corresponding
|
||||
Source of the work are being offered to the general public at no
|
||||
charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded
|
||||
from the Corresponding Source as a System Library, need not be
|
||||
included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any
|
||||
tangible personal property which is normally used for personal, family,
|
||||
or household purposes, or (2) anything designed or sold for incorporation
|
||||
into a dwelling. In determining whether a product is a consumer product,
|
||||
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||
product received by a particular user, "normally used" refers to a
|
||||
typical or common use of that class of product, regardless of the status
|
||||
of the particular user or of the way in which the particular user
|
||||
actually uses, or expects or is expected to use, the product. A product
|
||||
is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent
|
||||
the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods,
|
||||
procedures, authorization keys, or other information required to install
|
||||
and execute modified versions of a covered work in that User Product from
|
||||
a modified version of its Corresponding Source. The information must
|
||||
suffice to ensure that the continued functioning of the modified object
|
||||
code is in no case prevented or interfered with solely because
|
||||
modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or
|
||||
specifically for use in, a User Product, and the conveying occurs as
|
||||
part of a transaction in which the right of possession and use of the
|
||||
User Product is transferred to the recipient in perpetuity or for a
|
||||
fixed term (regardless of how the transaction is characterized), the
|
||||
Corresponding Source conveyed under this section must be accompanied
|
||||
by the Installation Information. But this requirement does not apply
|
||||
if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has
|
||||
been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a
|
||||
requirement to continue to provide support service, warranty, or updates
|
||||
for a work that has been modified or installed by the recipient, or for
|
||||
the User Product in which it has been modified or installed. Access to a
|
||||
network may be denied when the modification itself materially and
|
||||
adversely affects the operation of the network or violates the rules and
|
||||
protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided,
|
||||
in accord with this section must be in a format that is publicly
|
||||
documented (and with an implementation available to the public in
|
||||
source code form), and must require no special password or key for
|
||||
unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this
|
||||
License by making exceptions from one or more of its conditions.
|
||||
Additional permissions that are applicable to the entire Program shall
|
||||
be treated as though they were included in this License, to the extent
|
||||
that they are valid under applicable law. If additional permissions
|
||||
apply only to part of the Program, that part may be used separately
|
||||
under those permissions, but the entire Program remains governed by
|
||||
this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option
|
||||
remove any additional permissions from that copy, or from any part of
|
||||
it. (Additional permissions may be written to require their own
|
||||
removal in certain cases when you modify the work.) You may place
|
||||
additional permissions on material, added by you to a covered work,
|
||||
for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you
|
||||
add to a covered work, you may (if authorized by the copyright holders of
|
||||
that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the
|
||||
terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or
|
||||
author attributions in that material or in the Appropriate Legal
|
||||
Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or
|
||||
requiring that modified versions of such material be marked in
|
||||
reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or
|
||||
authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some
|
||||
trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that
|
||||
material by anyone who conveys the material (or modified versions of
|
||||
it) with contractual assumptions of liability to the recipient, for
|
||||
any liability that these contractual assumptions directly impose on
|
||||
those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further
|
||||
restrictions" within the meaning of section 10. If the Program as you
|
||||
received it, or any part of it, contains a notice stating that it is
|
||||
governed by this License along with a term that is a further
|
||||
restriction, you may remove that term. If a license document contains
|
||||
a further restriction but permits relicensing or conveying under this
|
||||
License, you may add to a covered work material governed by the terms
|
||||
of that license document, provided that the further restriction does
|
||||
not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you
|
||||
must place, in the relevant source files, a statement of the
|
||||
additional terms that apply to those files, or a notice indicating
|
||||
where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the
|
||||
form of a separately written license, or stated as exceptions;
|
||||
the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly
|
||||
provided under this License. Any attempt otherwise to propagate or
|
||||
modify it is void, and will automatically terminate your rights under
|
||||
this License (including any patent licenses granted under the third
|
||||
paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your
|
||||
license from a particular copyright holder is reinstated (a)
|
||||
provisionally, unless and until the copyright holder explicitly and
|
||||
finally terminates your license, and (b) permanently, if the copyright
|
||||
holder fails to notify you of the violation by some reasonable means
|
||||
prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is
|
||||
reinstated permanently if the copyright holder notifies you of the
|
||||
violation by some reasonable means, this is the first time you have
|
||||
received notice of violation of this License (for any work) from that
|
||||
copyright holder, and you cure the violation prior to 30 days after
|
||||
your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the
|
||||
licenses of parties who have received copies or rights from you under
|
||||
this License. If your rights have been terminated and not permanently
|
||||
reinstated, you do not qualify to receive new licenses for the same
|
||||
material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or
|
||||
run a copy of the Program. Ancillary propagation of a covered work
|
||||
occurring solely as a consequence of using peer-to-peer transmission
|
||||
to receive a copy likewise does not require acceptance. However,
|
||||
nothing other than this License grants you permission to propagate or
|
||||
modify any covered work. These actions infringe copyright if you do
|
||||
not accept this License. Therefore, by modifying or propagating a
|
||||
covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically
|
||||
receives a license from the original licensors, to run, modify and
|
||||
propagate that work, subject to this License. You are not responsible
|
||||
for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an
|
||||
organization, or substantially all assets of one, or subdividing an
|
||||
organization, or merging organizations. If propagation of a covered
|
||||
work results from an entity transaction, each party to that
|
||||
transaction who receives a copy of the work also receives whatever
|
||||
licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the
|
||||
Corresponding Source of the work from the predecessor in interest, if
|
||||
the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the
|
||||
rights granted or affirmed under this License. For example, you may
|
||||
not impose a license fee, royalty, or other charge for exercise of
|
||||
rights granted under this License, and you may not initiate litigation
|
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||
any patent claim is infringed by making, using, selling, offering for
|
||||
sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this
|
||||
License of the Program or a work on which the Program is based. The
|
||||
work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims
|
||||
owned or controlled by the contributor, whether already acquired or
|
||||
hereafter acquired, that would be infringed by some manner, permitted
|
||||
by this License, of making, using, or selling its contributor version,
|
||||
but do not include claims that would be infringed only as a
|
||||
consequence of further modification of the contributor version. For
|
||||
purposes of this definition, "control" includes the right to grant
|
||||
patent sublicenses in a manner consistent with the requirements of
|
||||
this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||
patent license under the contributor's essential patent claims, to
|
||||
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||
propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express
|
||||
agreement or commitment, however denominated, not to enforce a patent
|
||||
(such as an express permission to practice a patent or covenant not to
|
||||
sue for patent infringement). To "grant" such a patent license to a
|
||||
party means to make such an agreement or commitment not to enforce a
|
||||
patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license,
|
||||
and the Corresponding Source of the work is not available for anyone
|
||||
to copy, free of charge and under the terms of this License, through a
|
||||
publicly available network server or other readily accessible means,
|
||||
then you must either (1) cause the Corresponding Source to be so
|
||||
available, or (2) arrange to deprive yourself of the benefit of the
|
||||
patent license for this particular work, or (3) arrange, in a manner
|
||||
consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have
|
||||
actual knowledge that, but for the patent license, your conveying the
|
||||
covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that
|
||||
country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or
|
||||
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||
covered work, and grant a patent license to some of the parties
|
||||
receiving the covered work authorizing them to use, propagate, modify
|
||||
or convey a specific copy of the covered work, then the patent license
|
||||
you grant is automatically extended to all recipients of the covered
|
||||
work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within
|
||||
the scope of its coverage, prohibits the exercise of, or is
|
||||
conditioned on the non-exercise of one or more of the rights that are
|
||||
specifically granted under this License. You may not convey a covered
|
||||
work if you are a party to an arrangement with a third party that is
|
||||
in the business of distributing software, under which you make payment
|
||||
to the third party based on the extent of your activity of conveying
|
||||
the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory
|
||||
patent license (a) in connection with copies of the covered work
|
||||
conveyed by you (or copies made from those copies), or (b) primarily
|
||||
for and in connection with specific products or compilations that
|
||||
contain the covered work, unless you entered into that arrangement,
|
||||
or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting
|
||||
any implied license or other defenses to infringement that may
|
||||
otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot convey a
|
||||
covered work so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you
|
||||
to collect a royalty for further conveying from those to whom you convey
|
||||
the Program, the only way you could satisfy both those terms and this
|
||||
License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Use with the GNU Affero General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, you have
|
||||
permission to link or combine any covered work with a work licensed
|
||||
under version 3 of the GNU Affero General Public License into a single
|
||||
combined work, and to convey the resulting work. The terms of this
|
||||
License will continue to apply to the part which is the covered work,
|
||||
but the special requirements of the GNU Affero General Public License,
|
||||
section 13, concerning interaction through a network will apply to the
|
||||
combination as such.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of
|
||||
the GNU General Public License from time to time. Such new versions will
|
||||
be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Program specifies that a certain numbered version of the GNU General
|
||||
Public License "or any later version" applies to it, you have the
|
||||
option of following the terms and conditions either of that numbered
|
||||
version or of any later version published by the Free Software
|
||||
Foundation. If the Program does not specify a version number of the
|
||||
GNU General Public License, you may choose any version ever published
|
||||
by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future
|
||||
versions of the GNU General Public License can be used, that proxy's
|
||||
public statement of acceptance of a version permanently authorizes you
|
||||
to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different
|
||||
permissions. However, no additional obligations are imposed on any
|
||||
author or copyright holder as a result of your choosing to follow a
|
||||
later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided
|
||||
above cannot be given local legal effect according to their terms,
|
||||
reviewing courts shall apply local law that most closely approximates
|
||||
an absolute waiver of all civil liability in connection with the
|
||||
Program, unless a warranty or assumption of liability accompanies a
|
||||
copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
state the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If the program does terminal interaction, make it output a short
|
||||
notice like this when it starts in an interactive mode:
|
||||
|
||||
<program> Copyright (C) <year> <name of author>
|
||||
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||
This is free software, and you are welcome to redistribute it
|
||||
under certain conditions; type `show c' for details.
|
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||
parts of the General Public License. Of course, your program's commands
|
||||
might be different; for a GUI interface, you would use an "about box".
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||
For more information on this, and how to apply and follow the GNU GPL, see
|
||||
<https://www.gnu.org/licenses/>.
|
||||
|
||||
The GNU General Public License does not permit incorporating your program
|
||||
into proprietary programs. If your program is a subroutine library, you
|
||||
may consider it more useful to permit linking proprietary applications with
|
||||
the library. If this is what you want to do, use the GNU Lesser General
|
||||
Public License instead of this License. But first, please read
|
||||
<https://www.gnu.org/licenses/why-not-lgpl.html>.
|
||||
@@ -20,6 +20,9 @@ cp "$BIN" "$APP/Contents/MacOS/AutoZhuyin"
|
||||
cp Resources/Info.plist "$APP/Contents/Info.plist"
|
||||
cp Resources/lexicon.txt Resources/english-common.txt Resources/classifier-weights.json "$APP/Contents/Resources/"
|
||||
cp THIRD_PARTY_LICENSES.txt "$APP/Contents/Resources/"
|
||||
# "AI 選字" language model (CKIP gpt2-tiny-chinese, GPL-3.0), fetched and converted once.
|
||||
[ -s build/lm/lm.bin ] || scripts/fetch-lm.sh >/dev/null
|
||||
cp build/lm/lm.bin build/lm/lm-vocab.txt licenses/GPL-3.0.txt "$APP/Contents/Resources/"
|
||||
# Precomputed English dictionary and bigram table (shared with the Windows build).
|
||||
python3 -I windows/tools/build_data.py >/dev/null
|
||||
cp windows/autozhuyin/data/english-dict.txt windows/autozhuyin/data/english-bigram.json "$APP/Contents/Resources/"
|
||||
|
||||
Executable
+18
@@ -0,0 +1,18 @@
|
||||
#!/bin/bash
|
||||
# Downloads the CKIP gpt2-tiny-chinese language model (GPL-3.0,
|
||||
# https://huggingface.co/ckiplab/gpt2-tiny-chinese) and converts it to build/lm/:
|
||||
# lm.bin float16 weights (custom format, see training/lm/export_lm.py)
|
||||
# lm-vocab.txt
|
||||
# build.sh bundles them into the app when present ("AI 選字"). Safe to re-run.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
SRC=build/models/ckip-gpt2-tiny
|
||||
BASE="https://huggingface.co/ckiplab/gpt2-tiny-chinese/resolve/refs%2Fpr%2F1"
|
||||
mkdir -p "$SRC" build/lm
|
||||
for f in model.safetensors vocab.txt config.json; do
|
||||
[ -s "$SRC/$f" ] || curl -sfL -o "$SRC/$f" "$BASE/$f"
|
||||
done
|
||||
python3 -I training/lm/export_lm.py "$SRC" build/lm/lm.bin
|
||||
cp "$SRC/vocab.txt" build/lm/lm-vocab.txt
|
||||
ls -lh build/lm
|
||||
@@ -0,0 +1,147 @@
|
||||
"""Measures how much language-model reranking improves Zhuyin → Chinese conversion.
|
||||
|
||||
For each test sentence: turn the text into Dachen keystrokes (via the lexicon's
|
||||
readings), convert with the current Viterbi converter (baseline), take the K best
|
||||
paths, and rerank them with lexicon score + λ · LM log-probability.
|
||||
|
||||
Usage (needs numpy and the model in build/models/ckip-gpt2-tiny):
|
||||
python training/lm/eval_rerank.py build/lm-eval/chat.txt build/lm-eval/wiki.txt
|
||||
"""
|
||||
import heapq
|
||||
import math
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
sys.path.insert(0, os.path.join(ROOT, "windows"))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from autozhuyin.core import bopomofo # noqa: E402
|
||||
from autozhuyin.core.lexicon import Converter, Lexicon # noqa: E402
|
||||
from gpt2 import GPT2 # noqa: E402
|
||||
|
||||
KEY_OF = {sym: key for key, sym in bopomofo.SYMBOLS.items()}
|
||||
TONE_KEY = {"ˊ": "6", "ˇ": "3", "ˋ": "4", "˙": "7"}
|
||||
LN10 = math.log(10)
|
||||
|
||||
|
||||
def reading_keys(reading):
|
||||
out = []
|
||||
for syl in reading.split("-"):
|
||||
tone = " "
|
||||
for ch in syl:
|
||||
if ch in TONE_KEY:
|
||||
tone = TONE_KEY[ch]
|
||||
else:
|
||||
out.append(KEY_OF[ch])
|
||||
out.append(tone)
|
||||
return out
|
||||
|
||||
|
||||
def build_reverse(lex):
|
||||
best = {}
|
||||
for reading, entries in lex.exact.items():
|
||||
for phrase, score in entries:
|
||||
if phrase not in best or score > best[phrase][1]:
|
||||
best[phrase] = (reading, score)
|
||||
return best
|
||||
|
||||
|
||||
def text_to_keys(text, reverse, max_len):
|
||||
"""Most likely segmentation of the text into lexicon phrases → keystrokes."""
|
||||
n = len(text)
|
||||
best = [(-math.inf, None)] * (n + 1)
|
||||
best[0] = (0.0, None)
|
||||
for j in range(1, n + 1):
|
||||
for i in range(max(0, j - max_len), j):
|
||||
if best[i][0] == -math.inf:
|
||||
continue
|
||||
hit = reverse.get(text[i:j])
|
||||
if hit:
|
||||
s = best[i][0] + hit[1]
|
||||
if s > best[j][0]:
|
||||
best[j] = (s, (i, hit[0]))
|
||||
if best[n][1] is None:
|
||||
return None
|
||||
readings, j = [], n
|
||||
while j > 0:
|
||||
i, r = best[j][1]
|
||||
readings.append(r)
|
||||
j = i
|
||||
keys = []
|
||||
for r in reversed(readings):
|
||||
keys += reading_keys(r)
|
||||
return keys
|
||||
|
||||
|
||||
def kbest(conv, syllables, k):
|
||||
"""K best (score, text) paths through the converter's lattice (scores in log10)."""
|
||||
n = len(syllables)
|
||||
max_len = max(1, conv.lexicon.max_phrase_length)
|
||||
beams = [[] for _ in range(n + 1)]
|
||||
beams[0] = [(0.0, "")]
|
||||
for j in range(1, n + 1):
|
||||
cand = []
|
||||
for length in range(1, min(max_len, j) + 1):
|
||||
i = j - length
|
||||
if not beams[i]:
|
||||
continue
|
||||
for text, score in conv._options(syllables, i, j, {}):
|
||||
for s, t in beams[i]:
|
||||
cand.append((s + score, t + text))
|
||||
beams[j] = heapq.nlargest(k, cand, key=lambda e: e[0])
|
||||
return beams[n]
|
||||
|
||||
|
||||
def char_errors(a, b):
|
||||
return sum(x != y for x, y in zip(a, b)) + abs(len(a) - len(b))
|
||||
|
||||
|
||||
def main(paths, k=10):
|
||||
lex = Lexicon.from_file(os.path.join(ROOT, "windows/autozhuyin/data/lexicon.txt"))
|
||||
conv = Converter(lex)
|
||||
lm = GPT2(os.path.join(ROOT, "build/models/ckip-gpt2-tiny"))
|
||||
reverse = build_reverse(lex)
|
||||
|
||||
cases = []
|
||||
for path in paths:
|
||||
for line in open(path, encoding="utf-8"):
|
||||
text = line.strip().replace(",", "").replace("、", "")
|
||||
if not text:
|
||||
continue
|
||||
keys = text_to_keys(text, reverse, lex.max_phrase_length)
|
||||
if keys is None:
|
||||
continue
|
||||
syl = [s.syllable for s in bopomofo.parse(keys).syllables]
|
||||
paths_k = kbest(conv, syl, k)
|
||||
t0 = time.time()
|
||||
lm_scores = [lm.logprob(t) for _, t in paths_k]
|
||||
cases.append((os.path.basename(path), text, paths_k, lm_scores, time.time() - t0))
|
||||
|
||||
def evaluate(lam, subset):
|
||||
chars = errs = sent_ok = 0
|
||||
for _, text, paths_k, lm_scores, _ in subset:
|
||||
best = max(range(len(paths_k)), key=lambda i: paths_k[i][0] * LN10 + lam * lm_scores[i])
|
||||
out = paths_k[best][1]
|
||||
e = char_errors(out, text)
|
||||
chars += len(text)
|
||||
errs += e
|
||||
sent_ok += e == 0
|
||||
return 1 - errs / chars, sent_ok / len(subset)
|
||||
|
||||
lams = [0, 0.1, 0.2, 0.3, 0.5, 0.7, 1.0, 1.5, 2.0]
|
||||
for name in sorted({c[0] for c in cases}):
|
||||
sub = [c for c in cases if c[0] == name]
|
||||
oracle = sum(any(t == c[1] for _, t in c[2]) for c in sub) / len(sub)
|
||||
print(f"\n== {name}: {len(sub)} sentences, top-{k} contains the answer: {oracle:.1%}")
|
||||
for lam in lams:
|
||||
ca, sa = evaluate(lam, sub)
|
||||
print(f" λ={lam:<4} char acc {ca:.2%} sentence acc {sa:.1%}" + (" (baseline)" if lam == 0 else ""))
|
||||
ms = sum(c[4] for c in cases) / len(cases) * 1000
|
||||
print(f"\nLM time per sentence (top-{k}): {ms:.1f} ms")
|
||||
return cases
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1:])
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Converts the gpt2-tiny-chinese safetensors file to the input method's format.
|
||||
|
||||
Format (little-endian):
|
||||
"AZLM" magic, uint32 header length, JSON header, then float16 tensors back to back.
|
||||
Header: {"n_layer", "n_head", "n_embd", "vocab", "eps", "tensors": [[name, [shape], offset]]}
|
||||
(offset in float16 elements from the start of the data). Linear weights keep the
|
||||
GPT-2 Conv1D layout [in, out]. The attention mask buffers are dropped.
|
||||
|
||||
Usage: export_lm.py <model dir> <out.bin> (standard library only)
|
||||
"""
|
||||
import array
|
||||
import json
|
||||
import struct
|
||||
import sys
|
||||
|
||||
|
||||
def main(src, out):
|
||||
cfg = json.load(open(f"{src}/config.json"))
|
||||
with open(f"{src}/model.safetensors", "rb") as f:
|
||||
n = struct.unpack("<Q", f.read(8))[0]
|
||||
header = json.loads(f.read(n))
|
||||
base = 8 + n
|
||||
names = sorted(k for k, v in header.items() if k != "__metadata__" and v["dtype"] == "F32" and v["shape"])
|
||||
tensors, blobs, offset = [], [], 0
|
||||
for name in names:
|
||||
info = header[name]
|
||||
a, b = info["data_offsets"]
|
||||
f.seek(base + a)
|
||||
floats = array.array("f")
|
||||
floats.frombytes(f.read(b - a))
|
||||
if sys.byteorder != "little":
|
||||
floats.byteswap()
|
||||
half = struct.pack(f"<{len(floats)}e", *floats)
|
||||
tensors.append([name.replace("transformer.", ""), info["shape"], offset])
|
||||
blobs.append(half)
|
||||
offset += len(floats)
|
||||
meta = json.dumps({"n_layer": cfg["n_layer"], "n_head": cfg["n_head"], "n_embd": cfg["n_embd"],
|
||||
"vocab": cfg["vocab_size"], "eps": cfg["layer_norm_epsilon"],
|
||||
"tensors": tensors}).encode()
|
||||
with open(out, "wb") as f:
|
||||
f.write(b"AZLM" + struct.pack("<I", len(meta)) + meta)
|
||||
for blob in blobs:
|
||||
f.write(blob)
|
||||
print(f"wrote {out}: {offset} weights")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1], sys.argv[2])
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Minimal GPT-2 forward pass in numpy, for the CKIP gpt2-tiny-chinese model
|
||||
(https://huggingface.co/ckiplab/gpt2-tiny-chinese, GPL-3.0). Character-level:
|
||||
each Chinese character is one token of the bert-base-chinese vocabulary.
|
||||
|
||||
Used to evaluate language-model reranking before porting it to the input methods.
|
||||
"""
|
||||
import json
|
||||
import math
|
||||
import struct
|
||||
|
||||
import numpy as np
|
||||
|
||||
CLS, UNK = 101, 100
|
||||
|
||||
|
||||
def load_safetensors(path):
|
||||
with open(path, "rb") as f:
|
||||
n = struct.unpack("<Q", f.read(8))[0]
|
||||
header = json.loads(f.read(n))
|
||||
base = 8 + n
|
||||
out = {}
|
||||
for name, info in header.items():
|
||||
if name == "__metadata__" or info["dtype"] != "F32":
|
||||
continue
|
||||
a, b = info["data_offsets"]
|
||||
f.seek(base + a)
|
||||
out[name] = np.frombuffer(f.read(b - a), dtype=np.float32).reshape(info["shape"])
|
||||
return out
|
||||
|
||||
|
||||
class GPT2:
|
||||
def __init__(self, model_dir):
|
||||
self.w = load_safetensors(f"{model_dir}/model.safetensors")
|
||||
cfg = json.load(open(f"{model_dir}/config.json"))
|
||||
self.n_layer, self.n_head = cfg["n_layer"], cfg["n_head"]
|
||||
self.eps = cfg["layer_norm_epsilon"]
|
||||
self.vocab = {}
|
||||
for i, line in enumerate(open(f"{model_dir}/vocab.txt", encoding="utf-8")):
|
||||
self.vocab.setdefault(line.rstrip("\n"), i)
|
||||
|
||||
def ids(self, text):
|
||||
return [self.vocab.get(c, UNK) for c in text]
|
||||
|
||||
def _ln(self, x, name):
|
||||
mu = x.mean(-1, keepdims=True)
|
||||
var = ((x - mu) ** 2).mean(-1, keepdims=True)
|
||||
return (x - mu) / np.sqrt(var + self.eps) * self.w[name + ".weight"] + self.w[name + ".bias"]
|
||||
|
||||
def hidden(self, ids):
|
||||
"""Final hidden states [T, d] for token ids."""
|
||||
w, T = self.w, len(ids)
|
||||
x = w["transformer.wte.weight"][ids] + w["transformer.wpe.weight"][:T]
|
||||
mask = np.triu(np.full((T, T), -1e9, dtype=np.float32), 1)
|
||||
for l in range(self.n_layer):
|
||||
p = f"transformer.h.{l}."
|
||||
h = self._ln(x, p + "ln_1")
|
||||
qkv = h @ w[p + "attn.c_attn.weight"] + w[p + "attn.c_attn.bias"]
|
||||
q, k, v = np.split(qkv, 3, axis=-1)
|
||||
d = q.shape[-1] // self.n_head
|
||||
q = q.reshape(T, self.n_head, d).transpose(1, 0, 2)
|
||||
k = k.reshape(T, self.n_head, d).transpose(1, 0, 2)
|
||||
v = v.reshape(T, self.n_head, d).transpose(1, 0, 2)
|
||||
att = q @ k.transpose(0, 2, 1) / math.sqrt(d) + mask
|
||||
att = np.exp(att - att.max(-1, keepdims=True))
|
||||
att /= att.sum(-1, keepdims=True)
|
||||
a = (att @ v).transpose(1, 0, 2).reshape(T, -1)
|
||||
x = x + a @ w[p + "attn.c_proj.weight"] + w[p + "attn.c_proj.bias"]
|
||||
h = self._ln(x, p + "ln_2")
|
||||
h = h @ w[p + "mlp.c_fc.weight"] + w[p + "mlp.c_fc.bias"]
|
||||
h = 0.5 * h * (1 + np.tanh(math.sqrt(2 / math.pi) * (h + 0.044715 * h ** 3))) # gelu_new
|
||||
x = x + h @ w[p + "mlp.c_proj.weight"] + w[p + "mlp.c_proj.bias"]
|
||||
return self._ln(x, "transformer.ln_f")
|
||||
|
||||
def logprob(self, text, context=""):
|
||||
"""Natural-log probability of `text` given `context` (both strings)."""
|
||||
ctx, tgt = self.ids(context), self.ids(text)
|
||||
ids = [CLS] + ctx + tgt
|
||||
h = self.hidden(ids[:-1])
|
||||
logits = h @ self.w["transformer.wte.weight"].T
|
||||
logits -= logits.max(-1, keepdims=True)
|
||||
lse = np.log(np.exp(logits).sum(-1))
|
||||
start = len(ctx)
|
||||
return float(sum(logits[start + i, t] - lse[start + i] for i, t in enumerate(tgt)))
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "自動注音 AutoZhuyin",
|
||||
"version": "0.10.1",
|
||||
"version": "0.11.0",
|
||||
"guid": "{3895C6E3-6D40-4B4B-B4EA-CEEC75382D23}",
|
||||
"locale": "zh-Hant-TW",
|
||||
"fallbackLocale": "zh-TW",
|
||||
|
||||
Reference in New Issue
Block a user