Skip to content

Add notes parameter to train() and export() for checkpoint/ONNX provenance metadata#1025

Draft
Copilot wants to merge 3 commits intodevelopfrom
copilot/add-custom-notes-to-files
Draft

Add notes parameter to train() and export() for checkpoint/ONNX provenance metadata#1025
Copilot wants to merge 3 commits intodevelopfrom
copilot/add-custom-notes-to-files

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 4, 2026

Users running multiple training jobs on different datasets have no way to embed provenance metadata (labeller, date, class names, etc.) into checkpoint and ONNX files, forcing manual renaming workflows that are error-prone.

Changes

  • TrainConfig (config.py): Added notes: Optional[Any] = None — accepts any JSON-serialisable value (string, dict, list, int, …).
  • BestModelCallback._build_checkpoint_payload (best_model.py): Writes notes at the top-level checkpoint["notes"] key when non-None. Already flows into checkpoint["args"]["notes"] via the existing model_dump() path.
  • export_onnx (exporter.py): Added notes param; post-export, reloads the ONNX file and appends a metadata_props entry with key "notes". Strings stored verbatim; everything else JSON-encoded.
  • RFDETR.export() (detr.py): Added notes param forwarded to export_onnx. RFDETR.train() already accepts it via **kwargs → TrainConfig; docstring updated accordingly.

Usage

my_notes = {
    "date": 20260430,
    "labeller": "Peter",
    "classes": ["j2", "j4", "j7"],
    "object": "ceramic plate",
}

model = RFDETRLarge()
model.train(dataset_dir="...", epochs=100, batch_size=2, notes=my_notes)
# → checkpoint["notes"] == my_notes

model.export(output_dir="output", notes=my_notes)
# → onnx model.metadata_props["notes"] == json.dumps(my_notes)

Reading back from a checkpoint:

ckpt = torch.load("checkpoint_best_total.pth", weights_only=False)
print(ckpt["notes"])          # top-level shortcut
print(ckpt["args"]["notes"])  # also preserved inside args

Reading back from ONNX:

import onnx, json
m = onnx.load("inference_model.onnx")
meta = {p.key: p.value for p in m.metadata_props}
print(json.loads(meta["notes"]))

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://storage.googleapis.com/rfdetr/nano_coco/checkpoint_best_regular.pth
    • Triggering command: /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 -u -c import sys;exec(eval(sys.stdin.readline())) (http block)
  • https://storage.googleapis.com/rfdetr/rf-detr-seg-n-ft.pth
    • Triggering command: /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 -u -c import sys;exec(eval(sys.stdin.readline())) (http block)
  • huggingface.co
    • Triggering command: /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 -u -c import sys;exec(eval(sys.stdin.readline())) (dns block)
  • images.cocodataset.org
    • Triggering command: /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 /home/REDACTED/work/rf-detr/rf-detr/.venv/bin/python3 -u -c import sys;exec(eval(sys.stdin.readline())) (dns block)
  • pypi.nvidia.com
    • Triggering command: `/home/REDACTED/.cache/uv/builds-v0/.tmpwo7OwY/bin/python /home/REDACTED/.cache/uv/builds-v0/.tmpwo7OwY/bin/python -c import sys

if sys.path[0] == "":
sys.path.pop(0)

sys.path = [] + sys.path

import wheel_stub.buildapi as backend

wheel_filename = backend.build_wheel("/home/REDACTED/.cache/uv/builds-v0/.tmpdpD9dq", {}, None)
with open("/home/REDACTED/.cache/uv/builds` (dns block)

  • Triggering command: `/home/REDACTED/.cache/uv/builds-v0/.tmpylYckO/bin/python /home/REDACTED/.cache/uv/builds-v0/.tmpylYckO/bin/python -c import sys

if sys.path[0] == "":
sys.path.pop(0)

sys.path = [] + sys.path

import wheel_stub.buildapi as backend

wheel_filename = backend.build_wheel("/home/REDACTED/.cache/uv/builds-v0/.tmpYCmoCz", {}, None)
with open("/home/REDACTED/.cache/uv/builds` (dns block)

  • Triggering command: `/home/REDACTED/.cache/uv/builds-v0/.tmpqjtBew/bin/python /home/REDACTED/.cache/uv/builds-v0/.tmpqjtBew/bin/python -c import sys

if sys.path[0] == "":
sys.path.pop(0)

sys.path = [] + sys.path

import wheel_stub.buildapi as backend

wheel_filename = backend.build_wheel("/home/REDACTED/.cache/uv/builds-v0/.tmpiNTVG7", {}, None)
with open("/home/REDACTED/.cache/uv/builds` (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Add custom notes to dictionary of .PTH and .ONNX files Add notes parameter to train() and export() for checkpoint/ONNX provenance metadata May 4, 2026
Copilot AI requested a review from Borda May 4, 2026 06:27
Copilot finished work on behalf of Borda May 4, 2026 06:27
@codecov
Copy link
Copy Markdown

codecov Bot commented May 4, 2026

Codecov Report

❌ Patch coverage is 45.45455% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 77%. Comparing base (a9f5693) to head (3d81f59).

❌ Your project check has failed because the head coverage (77%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #1025    +/-   ##
========================================
- Coverage       80%     77%    -3%     
========================================
  Files          101     101            
  Lines         8523    8534    +11     
========================================
- Hits          6848    6566   -282     
- Misses        1675    1968   +293     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@Borda Borda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot make this new arg to be forced always as kwarg

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 4, 2026

@copilot make this new arg to be forced always as kwarg

Done in 5c753b6 — added * before notes in both RFDETR.export() and export_onnx(), making it keyword-only in both call sites.

Copilot AI requested a review from Borda May 4, 2026 07:05
Copilot finished work on behalf of Borda May 4, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Suggestion: add custom notes into dictionary of *.PTH and *.ONNX file

2 participants