o W hG@sddlmZddlZddlZddlmZddlmZm Z ddl m Z ddl m Z mZmZmZddlmZmZGdd d eZdS) )PathN)DetectionValidator)LOGGERops)check_requirements) OKS_SIGMA PoseMetricsbox_ioukpt_iou)output_to_target plot_imagescseZdZdZdfdd ZfddZddZfd d Zfd d Zfd dZ ddZ dddZ ddZ ddZ ddZddZddZZS) PoseValidatoraj A class extending the DetectionValidator class for validation based on a pose model. This validator is specifically designed for pose estimation tasks, handling keypoints and implementing specialized metrics for pose evaluation. Attributes: sigma (np.ndarray): Sigma values for OKS calculation, either OKS_SIGMA or ones divided by number of keypoints. kpt_shape (List[int]): Shape of the keypoints, typically [17, 3] for COCO format. args (dict): Arguments for the validator including task set to "pose". metrics (PoseMetrics): Metrics object for pose evaluation. Methods: preprocess: Preprocesses batch data for pose validation. get_desc: Returns description of evaluation metrics. init_metrics: Initializes pose metrics for the model. _prepare_batch: Prepares a batch for processing. _prepare_pred: Prepares and scales predictions for evaluation. update_metrics: Updates metrics with new predictions. _process_batch: Processes batch to compute IoU between detections and ground truth. plot_val_samples: Plots validation samples with ground truth annotations. plot_predictions: Plots model predictions. save_one_txt: Saves detections to a text file. pred_to_json: Converts predictions to COCO JSON format. eval_json: Evaluates model using COCO JSON format. Examples: >>> from ultralytics.models.yolo.pose import PoseValidator >>> args = dict(model="yolo11n-pose.pt", data="coco8-pose.yaml") >>> validator = PoseValidator(args=args) >>> validator() Ncsjt|||||d|_d|_d|j_t|jd|_t |jj t r1|jj dkr3t ddSdSdS)a Initialize a PoseValidator object for pose estimation validation. This validator is specifically designed for pose estimation tasks, handling keypoints and implementing specialized metrics for pose evaluation. Args: dataloader (torch.utils.data.DataLoader, optional): Dataloader to be used for validation. save_dir (Path | str, optional): Directory to save results. pbar (Any, optional): Progress bar for displaying progress. args (dict, optional): Arguments for the validator including task set to "pose". _callbacks (list, optional): List of callback functions to be executed during validation. Examples: >>> from ultralytics.models.yolo.pose import PoseValidator >>> args = dict(model="yolov8n-pose.pt", data="coco8-pose.yaml") >>> validator = PoseValidator(args=args) >>> validator() Notes: This class extends DetectionValidator with pose-specific functionality. It initializes with sigma values for OKS calculation and sets up PoseMetrics for evaluation. A warning is displayed when using Apple MPS due to a known bug with pose models. Npose)save_dirmpsz}Apple MPS known Pose bug. Recommend 'device=cpu' for Pose models. See https://github.com/ultralytics/ultralytics/issues/4031.)super__init__sigma kpt_shapeargstaskrrmetrics isinstancedevicestrlowerrwarning)self dataloaderrpbarr _callbacks __class__T/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/models/yolo/pose/val.pyr1szPoseValidator.__init__cs(t|}|d|j|d<|S)zSPreprocess batch by converting keypoints data to float and moving it to the device. keypoints)r preprocesstorfloat)rbatchr!r#r$r&Us zPoseValidator.preprocesscCsddS)z:Return description of evaluation metrics in string format.z,%22s%11s%11s%11s%11s%11s%11s%11s%11s%11s%11s) ClassImages InstanceszBox(PRmAP50 mAP50-95)zPose(Pr-r.r/r#rr#r#r$get_desc[szPoseValidator.get_desccsbt||jd|_|jddgk}|jd}|rtnt|||_tggggggd|_ dS)z2Initialize pose estimation metrics for YOLO model.rr)tp_ptpconfpred_cls target_cls target_imgN) r init_metricsdatarrnponesrdictstats)rmodelis_posenkptr!r#r$r:ks   zPoseValidator.init_metricscst||}|d|d|k}|d\}}|}|d|9<|d|9<tj|d||d|dd}||d <|S) aS Prepare a batch for processing by converting keypoints to float and scaling to original dimensions. Args: si (int): Batch index. batch (dict): Dictionary containing batch data with keys like 'keypoints', 'batch_idx', etc. Returns: pbatch (dict): Prepared batch with keypoints scaled to original image dimensions. Notes: This method extends the parent class's _prepare_batch method by adding keypoint processing. Keypoints are scaled from normalized coordinates to original image dimensions. r% batch_idximgsz).r). ori_shape ratio_padrGkpts)r_prepare_batchcloner scale_coords)rsir)pbatchrIhwr!r#r$rJts zPoseValidator._prepare_batchcsdt||}|djd}|ddddft||d}tj|d||d|dd ||fS) aC Prepare and scale keypoints in predictions for pose processing. This method extends the parent class's _prepare_pred method to handle keypoint scaling. It first calls the parent method to get the basic prediction boxes, then extracts and scales the keypoint coordinates to match the original image dimensions. Args: pred (torch.Tensor): Raw prediction tensor from the model. pbatch (dict): Processed batch dictionary containing image information including: - imgsz: Image size used for inference - ori_shape: Original image shape - ratio_pad: Ratio and padding information for coordinate scaling Returns: predn (torch.Tensor): Processed prediction boxes scaled to original image dimensions. rIrENrDrFrGrH)r _prepare_predshapeviewlenrrL)rpredrNprednnk pred_kptsr!r#r$rSs "zPoseValidator._prepare_predc Cs"t|D] \}}|jd7_t|}ttjd|jdtjd|jdtj||jtj|jdtj||jtj|jdd}| ||}| d| d}} t|} ||d<| |d <|dkr| r|j D] } |j | || qf|jjr|jjd | |d q|jjrd|d d d f<|||\} } | d d d f|d<| d d d f|d<| r|| | ||d<|| | || |d|d<|jjr|j| | ||j D] } |j | || q|jjr|| |d||jjr|| | |jj|d|jdt|d|jdqd S)a Update metrics with new predictions and ground truth data. This method processes each prediction, compares it with ground truth, and updates various statistics for performance evaluation. Args: preds (List[torch.Tensor]): List of prediction tensors from the model. batch (dict): Batch data containing images and ground truth annotations. rEr)r)dtyper)r6r7r5r4clsbboxr8r9N) detections gt_bboxesgt_clsr6r7r5rIr4im_filerFlabelsz.txt) enumerateseenrVr>torchzerosrniouboolrJpopuniquer?keysappendrplotsconfusion_matrix process_batch single_clsrS_process_batch save_json pred_to_jsonsave_txt save_one_txt save_confrrstem)rpredsr)rMrWnprstatrNr\r]nlkrXrZr#r#r$update_metricssZ     zPoseValidator.update_metricscCs||dur$|dur$t|ddddfdd}t|||j|d}n t||ddddf}||dddf||S)a Return correct prediction matrix by computing Intersection over Union (IoU) between detections and ground truth. Args: detections (torch.Tensor): Tensor with shape (N, 6) representing detection boxes and scores, where each detection is of the format (x1, y1, x2, y2, conf, class). gt_bboxes (torch.Tensor): Tensor with shape (M, 4) representing ground truth bounding boxes, where each box is of the format (x1, y1, x2, y2). gt_cls (torch.Tensor): Tensor with shape (M,) representing ground truth class indices. pred_kpts (torch.Tensor | None): Optional tensor with shape (N, 51) representing predicted keypoints, where 51 corresponds to 17 keypoints each having 3 values. gt_kpts (torch.Tensor | None): Optional tensor with shape (N, 51) representing ground truth keypoints. Returns: (torch.Tensor): A tensor with shape (N, 10) representing the correct prediction matrix for 10 IoU levels, where N is the number of detections. Notes: `0.53` scale factor used in area computation is referenced from https://github.com/jin-s13/xtcocoapi/blob/master/xtcocotools/cocoeval.py#L384. NrEg(\?)rarearbra)r xyxy2xywhprodr rr match_predictions)rr^r_r`rZgt_kptsriour#r#r$rss $zPoseValidator._process_batchc CsNt|d|d|dd|d|d|d|jd|d |j|jd d S) am Plot and save validation set samples with ground truth bounding boxes and keypoints. Args: batch (dict): Dictionary containing batch data with keys: - img (torch.Tensor): Batch of images - batch_idx (torch.Tensor): Batch indices for each image - cls (torch.Tensor): Class labels - bboxes (torch.Tensor): Bounding box coordinates - keypoints (torch.Tensor): Keypoint coordinates - im_file (list): List of image file paths ni (int): Batch index used for naming the output file imgrCr\rRbboxesr%rc val_batchz _labels.jpgrIpathsfnamenameson_plotN)r squeezerrr)rr)nir#r#r$plot_val_sampless  zPoseValidator.plot_val_samplescsbtfdd|Dd}t|dgt|jjdR||djd|djjd d S) a Plot and save model predictions with bounding boxes and keypoints. Args: batch (dict): Dictionary containing batch data including images, file paths, and other metadata. preds (List[torch.Tensor]): List of prediction tensors from the model, each containing bounding boxes, confidence scores, class predictions, and keypoints. ni (int): Batch index used for naming the output file. The function extracts keypoints from predictions, converts predictions to target format, and plots them on the input images. The resulting visualization is saved to the specified save directory. cs0g|]}|ddddfjdgjRqS)NrQrR)rUr).0pr0r#r$ )s0z2PoseValidator.plot_predictions..rr)max_detrcrz _pred.jpgrN) rgcatr r rrrrr)rr)rzrrZr#r0r$plot_predictionss  zPoseValidator.plot_predictionscCsVddlm}|tj|d|dftjdd|j|ddddf|dj||ddS) a Save YOLO pose detections to a text file in normalized coordinates. Args: predn (torch.Tensor): Prediction boxes and scores with shape (N, 6) for (x1, y1, x2, y2, conf, cls). pred_kpts (torch.Tensor): Predicted keypoints with shape (N, K, D) where K is the number of keypoints and D is the dimension (typically 3 for x, y, visibility). save_conf (bool): Whether to save confidence scores. shape (tuple): Original image shape (height, width). file (Path): Output file path to save detections. Notes: The output format is: class_id x_center y_center width height confidence keypoints where keypoints are normalized (x, y, visibility) values for each point. r)ResultsrE)r[NrQ)pathrboxesr%)rx)ultralytics.engine.resultsrr<rhuint8rrv)rrXrZrxrTfilerr#r#r$rw4s  zPoseValidator.save_one_txtc Cst|j}|r t|n|}t|ddddf}|ddddf|ddddfd8<t||D]%\}}|j ||j t|ddd|D|ddt |dddq=dS) a Convert YOLO predictions to COCO JSON format. This method takes prediction tensors and a filename, converts the bounding boxes from YOLO format to COCO format, and appends the results to the internal JSON dictionary (self.jdict). Args: predn (torch.Tensor): Prediction tensor containing bounding boxes, confidence scores, class IDs, and keypoints, with shape (N, 6+K) where N is the number of predictions and K is the flattened keypoints dimension. filename (str | Path): Path to the image file for which predictions are being processed. Notes: The method extracts the image ID from the filename stem (either as an integer if numeric, or as a string), converts bounding boxes from xyxy to xywh format, and adjusts coordinates from center to top-left corner before saving to the JSON dictionary. NrbrracSsg|]}t|dqS)r3)roundrxr#r#r$risz.PoseValidator.pred_to_json..rQ)image_id category_idr]r%score) rry isnumericintrrziptolistjdictrn class_mapr)rrXfilenameryrboxrbr#r#r$ruNs 0   zPoseValidator.pred_to_jsonc Cs~|jjr|jrt|jr|jdd}|jd}td|d|dz}t ddd l m }dd l m }||fD] }|sFJ|d q9|t|}|t|}t|||d |||d gD]<\} } |jrudd|jjjD| j_| | | | dd} | jdd\||jj| d<||jj| <qbW|Sty} ztd| WYd} ~ |Sd} ~ ww|S)z7Evaluate object detection model using COCO JSON format.rz)annotations/person_keypoints_val2017.jsonzpredictions.jsonz" Evaluating pycocotools mAP using z and z...zpycocotools>=2.0.6r)COCO)COCOevalz file not foundr]r%cSsg|] }tt|jqSr#)rrryrr#r#r$rsz+PoseValidator.eval_json..rbrNrEzpycocotools unable to run: )rrtis_cocorVrr;rrinforpycocotools.cocorpycocotools.cocoevalris_filerloadResrerdatasetim_filesparamsimgIdsevaluate accumulate summarizer?rrm Exceptionr) rr? anno_json pred_jsonrrrannorWievalidxer#r#r$ eval_jsonos<     $ & zPoseValidator.eval_json)NNNNN)NN)__name__ __module__ __qualname____doc__rr&r1r:rJrSrrsrrrwrur __classcell__r#r#r!r$r s! $   >!r )pathlibrnumpyr<rgultralytics.models.yolo.detectrultralytics.utilsrrultralytics.utils.checksrultralytics.utils.metricsrrr r ultralytics.utils.plottingr r r r#r#r#r$s