okayuji/indra-ocr-cjk-coreai-fp32

Indra OCR CJK recognizer for Core AI

Run the Japanese, Chinese and Korean text-line recognition component of Indra OCR with Apple's Core AI runtime. This research package returns character scores for ten already prepared text lines.

It contains one fixed-shape FP32 neural component. Page detection, layout, orientation, script routing and language-model decoding are separate parts of the original OCR pipeline.

For developers testing the Core AI conversion or building their own line-recognition host. This package does not accept document images directly. Requires an Apple-silicon Mac with macOS 27 or later and Python 3.11.

Quick start

python3 -m venv indra-env
source indra-env/bin/activate
pip install "coreai-core==1.0.0b3" "numpy==2.4.6" "huggingface_hub==2.4.0"
hf download okayuji/indra-ocr-cjk-coreai-fp32 --local-dir indra-cjk
python indra-cjk/run_cjk.py indra-cjk

The last command checks the files listed in manifest.json and runs a blank-input smoke test. To read your own prepared lines, pass both --pixels lines.f32 and --widths widths.i32; their exact format is below.

Sample output

{
  "logits_shape": [
    10,
    132,
    11750
  ],
  "finite": true,
  "decode": "greedy CTC only; no language model or confidence filtering",
  "text": [
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    "",
    ""
  ]
}

This is the recorded blank-input ABI test, not a page-OCR quality example. The run uses the same run_cjk.py shipped here. It performs greedy CTC decoding only; language-model beam search and confidence filtering are not included.

Verification

Original conversion check: 2026-11-01 JST. Release-asset and smoke checks: 2026-11-02 JST. Measured on Apple M5 Max / Mac17,6, 128 GiB RAM, macOS 27.2 (26B123), Python 3.11.9, torch 2.9.0, coreai-torch 0.4.3, coreai-core 1.0.0b3 and ONNX Runtime 1.31.1. The Core AI OS runtime used default specialization settings, one job at a time. The original FP32 ONNX recognizer is the numerical reference.

Three value conditions share the fixed shape: a captured batch from a synthetic Japanese page, zero ink, and the same batch at 0.75 intensity. For the original conversion, logit nRMSE was 3.63e-7 to 8.34e-7 against a predeclared maximum of 1e-4. Greedy token IDs, valid lengths and top-eight token order matched. See verification.json for unrounded results.

Debug locations were removed through the Core AI authoring API before packaging. A generic-IR digest including every constant and resource matched before stripping, after stripping, and after reloading the release asset. All three test outputs were bitwise identical between the original and released assets. No model weight was quantized or retrained.

Whole-page quality and timing measured with the original Indra ONNX pipeline are not measurements of this component. No component speed benchmark or minimum-memory claim is made here.

Input and output contract

The main function takes two float32 tensors:

Name	Shape	Meaning
pixels	[10, 1, 32, 528]	Text-line ink maps, 0 for paper and 1 for ink, right-padded with zero
mask	[10, 1, 132]	One at valid timesteps, zero after each line's valid length
Output logits	[10, 132, 11750]	Unnormalized CTC character scores

The host computes lengths = min((widths + 3) // 4, 132) and mask = arange(132) < lengths. Class zero is the CTC blank. Class i > 0 maps to vocab.json["chars"][i - 1]. Truncate to the valid length, collapse consecutive equal IDs, remove blanks, join the characters and normalize to NFC.

The upstream line preprocessing expands each detected ink box by 0.2 line heights above/below and 0.3 on the left/right, resizes to height 32 with aspect ratio preserved, and converts grayscale to an ink map. Use the fixed upstream crop and padding rules. Arbitrarily stretching a page into this tensor is not equivalent preprocessing.

Custom input to the helper is a raw little-endian float32 file with exactly 10*1*32*528 values and a raw little-endian int32 file with ten original pre-padding widths in 1..528. The helper rejects wrong shapes, non-finite or out-of-range ink, and invalid widths. It does not implement line detection or accept PNG/PDF files. See abi.json.

Files

• rec_cjk_fp32.aimodel/: the Core AI neural component, 10,181,240 bytes.
• vocab.json: the upstream CJK character vocabulary.
• run_cjk.py: executed hash-checking and greedy-CTC example.
• abi.json, manifest.json, verification.json: input/output contract, file identities and numerical evidence.
• LICENSE, NOTICE: model terms and source/change attribution.
• LICENSE-CODE: license for the independently authored helper code.

Operational notes and limitations

The first load/call can include specialization. File size is not process memory, and the asset size does not establish an iPhone memory budget. The Python runtime and this example have been tested on the Mac environment above; iPhone and Swift integration are untested.

Only the fixed tensor profile is supported. It has not been validated for variable batch, height or width, arbitrary page images, PDFs, vertical-writing quality, complete document layout, table reconstruction, long-lived resource release or cancellation. The remaining seven upstream neural components and the n-gram language models are not included. This component cannot be substituted for the complete indra-ocr package.

If the helper reports Payload verification failed, download the same revision again into a separate folder and compare its manifest. If it reports pixels must be float32 with shape [10, 1, 32, 528], check the raw input layout and shape rather than reshaping a document image directly.

License and attribution

Derived from Datalab's Indra OCR v1.1.0, CJK recognizer rec_cjk.onnx; source-code reference 3d4eb01.

The model, vocabulary, derived model assets and model outputs use the upstream AI PUBS OPEN RAIL-M LICENSE (MODIFIED), copied in full in LICENSE. Downloading, reproducing, distributing or using them is subject to that license, including paragraph 5 and Attachment A's use restrictions, attribution and share-alike terms. It contains commercial and competitive-use restrictions; it is not the standard Apache-2.0 license used by the upstream inference code. Read the complete terms before using or redistributing the model.

NOTICE identifies the conversion, the fixed host/neural split and the debug-information removal. The independently written helper code uses LICENSE-CODE; this does not alter the model or output terms. Indra, Datalab and Apple names identify upstream work and the target platform, without implying endorsement.