happyinhappy/nafnet-jewelry-restore

NAFNet Jewelry Restore

Takes a jewellery product shot that has been dusted, haired, scratched, blurred, noised, compressed and halved in resolution, and gives back the clean frame. Denoise, defect removal and detail recovery in one pass, trained on jewellery specifically rather than on a generic restoration set.

The weights are gated. The files in this repository are released on request: ask for access and the owner approves it by hand. The card itself, and the figures in it, stay public - the figures are served from happyin.work precisely so that gating the weights does not blank the page for someone deciding whether to ask.

What is in it: the iter-80,000 weights, both the EMA and the live copy, the run config and the run's own evaluation sheet for that checkpoint - alongside this card's account of how the data was built, how it was trained and what the checkpoints actually did over training.

Examples

Real evaluation output from the run's own sheets. Three columns: the clean reference, the deliberately degraded input, and what the model produced from it.

Read the middle column closely — that is where the dust specks and the stray hair across the pearl are. They are gone on the right, and the pearl's specular roll-off and the chain's facets survive. That is the difficulty of this domain in one frame: the defects and the product detail live at the same spatial frequency, so anything that removes one by blurring also removes the other.

Why jewellery needs its own restorer

A generic denoiser trained on faces and landscapes learns that fine high-frequency structure is usually noise. On a jewellery shot it is the product: facet edges, the grain of brushed metal, the exact roll-off of a specular highlight on a pearl. Remove it and you have not cleaned the photo, you have deleted what the customer is buying.

Meanwhile the actual defects — dust on the set, a hair fallen across the piece, sensor noise from a stopped-down macro exposure — sit right next to that structure.

Data: the pairs are manufactured

There is no archive of "damaged jewellery photographs with clean counterparts", so the training pairs are made rather than collected. The target is a real jewellery frame. The input is that same frame put through a degradation generator: dust, hairs, sensor noise, blur, colour noise and JPEG, then dropped to half resolution. Nothing is paired by hand and nothing is scraped — one clean photograph produces an unlimited number of damaged versions of itself, and the model is asked to undo whatever was done.

Panels 1–7 are the real generator run on one clean crop, one family at a time. Panel 8 is the whole chain composed and halved — the actual training input. Panels 9 and 10 are the run's own held-out evaluation frame and what the checkpoint made of it.

The chain, in the order it is applied

Every sample is degraded on the fly inside the dataloader; nothing is pre-rendered, so no two epochs see the same damage.

1. Structured defects. Sub-pixel dust specks — 700 to 2,400 of them, about 70 % clustered around 3–9 random centres and the rest sprinkled evenly, coloured dark, light or brown-dirt, mostly semi-transparent (alpha 0.06–0.32), softened to round grains with a 0.7–1.1 px blur. On top: 8 to 24 wavy hairs and lint fibres, 1–2 px wide, drawn as drifting curves with alpha 0.12–0.45, dark for 60 % of them and blonde for the rest, and on 38 % of samples multiplied by a further 1.7–3.2× for a genuinely hairy frame; 2–6 hard scratches; 14–60 defocused dust donuts; 1–4 broad grime blooms. Every one of those counts is scaled by the run's density of 0.85. A free defect-location mask falls out of this step, which is what makes it worth stamping the defects on their own layer.
2. Film damage, on a quarter of samples: vertical gate scratches, dirt "boil" blotches, heavy monochrome grain, vignette and a brightness flicker.
3. A generic camera chain, applied once, and twice on 30 % of samples. Its four steps are shuffled each time, so the model never learns one fixed order:
   • blur — one of isotropic Gaussian (σ ≤ 1.6), motion blur (trail ≤ 14 px, random angle) or selective focus, where a sharp band or disc roams the frame and blur ramps away from it; 28 % of draws get no blur at all;
   • resize round-trip — down to 0.65–0.95× and back, resampler picked at random from bilinear, bicubic and nearest;
   • sensor noise — shot and read noise applied in linear light after an inverse gamma, shot ≤ 0.010 and read ≤ 0.018, monochrome on 40 % of draws and per-channel otherwise.
4. Chroma noise on 30 % of samples: blotchy low-frequency noise injected into Cb and Cr only, σ 0.02–0.10, correlation length 1.0–3.5 px — the coloured speckle of a high-ISO sensor rather than clean white noise.
5. Chromatic aberration on 25 % of samples: red and blue channels shifted up to 1.5 px against green.
6. JPEG, always, at quality 45–92.
7. Half resolution. The degraded frame is Lanczos-downscaled by 2 to become the input; the target stays at full size. A 256 px clean crop yields a 128 px input.
8. Neural film grain, on 55 % of batches, added to the input on the GPU by GrainNet at strength 0.03–0.15 — real scanned-film grain statistics rather than Gaussian noise.

Half of all samples are instead drawn from a gentler preset (blur ≤ 1.0, shot ≤ 0.006, JPEG 55–95, defect density 0.4), so the model spans lightly-marked to heavily-damaged rather than only ever seeing the worst case. After 60,000 iterations the whole schedule switches to a harder one — blur to σ 3.0, motion to 20 px, noise roughly doubled, JPEG down to 22, defect density 0.95 — a curriculum that raises the difficulty only once the network has converged on the easy range.

Where the code is

The generator is src/degrade.py (2,341 lines) in the denoise-lab-deepcolor-20260704 project; the pairing lives in src/dataset.py and the grain and curriculum in src/train.py.

Step	Function	Location
Chain entry point	synthesize()	src/degrade.py:1878
Dust specks	_stamp_specks()	src/degrade.py:1719
Hairs and fibres	_fiber()	src/degrade.py:1601
Scratches	_scratch()	src/degrade.py:1648
Donuts, grime, compositing, mask	stamp_defects()	src/degrade.py:1750
Film damage	film_damage()	src/degrade.py:1836
Blur selector	add_blur()	src/degrade.py:1523
Selective focus	selective_focus()	src/degrade.py:1463
Sensor noise (linear light)	add_sensor_noise()	src/degrade.py:49
Chroma noise	add_chroma_noise()	src/degrade.py:66
Chromatic aberration	chromatic_shift()	src/degrade.py:1542
Resize round-trip	random_resize()	src/degrade.py:1555
JPEG	jpeg()	src/degrade.py:1566
Gentle preset	_MILD_DEFAULT	src/degrade.py:31
Degrade → downscale → pair	SyntheticPairDataset.__getitem__	src/dataset.py:315-322
GrainNet grain on the input	_add_grain()	src/train.py:494
Curriculum switch	_degrade_for()	src/train.py:356

Every number above comes from configs/nafnet_w64_jewelry_sr.yaml (run_tag: jewelry_sr, NAFNet-w64, scale: 2, 80,000 iterations, checkpoint every 2,000) — the config that produced this model's data. Its sibling nafnet_w64_jewelry_x4.yaml is a later, ×4 generation which adds uneven noise fields, a "sand sprinkled on the photo" grit layer and four degradations of each crop per batch; those are not in this model's training data.

degrade.py also carries roughly twenty further defect families — posterisation banding, surface smudges, embedded debris shards, white ring residue, glossy scratch fibres, triangular flakes, sharpened point clusters and others. All of them default to probability zero and this run's config does not switch any of them on, so they contributed nothing here. The texture defect type is likewise inert: it composites real film-scan overlays from data/textures/, and that directory does not exist on the training volume.

What the model can and cannot have learned

The degradations are ordered so the pair stays a valid restoration problem: defects are stamped before the blur and resize, so a speck sitting under a blur is blurred like a real speck would be, instead of being pasted crisply onto an already-soft frame. Colour is displaced only in ways that a lens or sensor displaces it, so luminance stays registered between input and target.

The consequence is the honest limit of the model: it has seen dust, lint, hair, scratches, grime, film damage, defocus, camera shake, sensor and chroma noise, colour fringing, JPEG and resolution loss. It has not seen anything else. Fingerprints, liquid marks, torn backdrops, motion of the product itself and compression artefacts from codecs other than JPEG are outside the distribution, and the model's behaviour on them is undefined rather than merely weaker.

Training and convergence

The run is 80,000 iterations, checkpointed every 2,000 — forty checkpoints, with an evaluation sheet rendered per checkpoint over the same held-out set. Progress is read off those sheets: each is a contact sheet of clean / degraded / restored triples on identical inputs, so two checkpoints are compared by eye on the same frames rather than by a single scalar. An earlier version of this card listed only iter 2000 … 14000 and called that the convergence record; those are the first seven sheets of the run, not the run.

Architecture: NAFNet — a restoration network that drops attention and activation functions in favour of simple gated blocks, which is why it holds fine texture instead of smoothing it.

Limitations

• Trained for jewellery. On skin, fabric or landscape it will behave like a network that thinks every fine detail is a facet.
• Synthetic degradation. Dust, hair and downscale were applied programmatically; real defects that do not resemble those three are out of distribution.
• It is a restorer, not a retoucher. It does not remove a reflection you dislike or reshape a stone — it undoes damage.
• Large frames need tiling. Product shots run 6K–10K px; the pipeline tiles them, and the tiling is where seams have to be managed (see the write-up on tiles with memory at https://happyin.work/blog/diffusion-seams-40mp/ ).

Contact

Anastasiia Butova — ML engineer, diffusion models and image processing. https://happyin.work/ois-gold/ · https://github.com/AnastasiyaW · https://t.me/happy_in_happy