Example 15: Use RF-DETR detection and SORT tracking

Install pyppbox, requirements/rfdetr.txt, and the optional pyppbox-data-rfdetr wheel as described in 🚀 Getting Started. From the repository’s examples/ directory:

python example_15_rfdetr.py --device cpu --frames 30

Use --device cuda:0 for NVIDIA GPU inference, --show for a video window, or --video path/to/video.mp4 for another clip. ReID is disabled in this example.

Configuration

RF_DETR supports Nano, Small, Medium, and Large detection checkpoints. The default is Nano on GPU 0, using a local model file:

dt_name: RF_DETR
model_size: nano
model_file: data/modules/rf_detr/rf-detr-nano.pth
device: 0
conf: 0.4
max_det: 100
person_class_id: 1
optimize: True
repspoint_calibration: 0.25

Select a checkpoint matching model_size. Set person_class_id to the native person class ID for a custom model; the supplied COCO checkpoints use 1. optimize prepares the model for inference. See Configurations for path and device rules.

Downloading pretrained weights

The data wheel supplies these files under data/modules/rf_detr/ in the installed package. For manual setup, download a checkpoint and select it in the RF-DETR settings panel.

Pretrained RF-DETR detection checkpoints

model_size	Resolution	Save as	Download
nano	384	rf-detr-nano.pth	Nano checkpoint
small	512	rf-detr-small.pth	Small checkpoint
medium	576	rf-detr-medium.pth	Medium checkpoint
large	704	rf-detr-large-2026.pth	Large checkpoint

Nano, Small, and Medium share the server filename checkpoint_best_regular.pth; rename each download as shown above. Set model_file: '' to let RF-DETR download and cache weights automatically (RF_HOME controls the cache). For offline use, install the data wheel or select a local checkpoint in advance.

Example source

#################################################################################
# Example of using RF-DETR detection and SORT tracking
#################################################################################

"""Run from examples/: python example_15_rfdetr.py --device cpu --frames 30.

Install requirements/rfdetr.txt from the repository root first.
Pass --checkpoint for local weights;
an empty checkpoint uses RF-DETR's download/cache. Add --show for a video window.
"""

import argparse
from pathlib import Path

import cv2

from pyppbox.ppb.mt import MT
from pyppbox.utils.visualizetools import visualizePeople


def main():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument('--video', default=str(Path(__file__).parent / 'data/gta.mp4'))
    parser.add_argument('--model-size', choices=['nano', 'small', 'medium', 'large'], default='nano')
    parser.add_argument('--checkpoint', default='', help='Optional local checkpoint matching --model-size.')
    parser.add_argument('--person-class-id', type=int, default=1, help='COCO person ID is 1; custom models may use 0.')
    parser.add_argument('--device', default='cpu', help='cpu, cuda:0, mps, or auto')
    parser.add_argument('--frames', type=int, default=30)
    parser.add_argument('--show', action='store_true', help='Display annotated frames in an OpenCV window.')
    args = parser.parse_args()
    if args.frames < 1:
        parser.error('--frames must be positive')
    capture = cv2.VideoCapture(args.video)
    try:
        if not capture.isOpened():
            raise FileNotFoundError(f'Cannot open video: {args.video}')
        # Set a detector by dictionary and the other stages by their names.
        with MT() as pipeline:
            pipeline.setMainDetector(dict(dt_name='RF_DETR', model_size=args.model_size,
                                          model_file=args.checkpoint, person_class_id=args.person_class_id,
                                          device=args.device, conf=.4))
            pipeline.setMainTracker('SORT')
            pipeline.setMainReIDer('None')
            for index in range(args.frames):
                ok, frame = capture.read()
                if not ok:
                    break
                # Detect and track Person objects, as in the earlier examples.
                people, _ = pipeline.detectPeople(frame, img_is_mat=True)
                people = pipeline.trackPeople(frame, people, img_is_mat=True)
                print(f'Frame {index}: {len(people)} people; track IDs {[person.cid for person in people]}')
                if args.show:
                    cv2.imshow('RF-DETR + SORT', visualizePeople(frame, people))
                    if cv2.waitKey(1) & 0xFF == ord('q'):
                        break
    finally:
        capture.release()
        if args.show:
            cv2.destroyAllWindows()


if __name__ == '__main__':
    main()