o Wñ hEQã@s ddlZddlmZddlmZddlmZddlmZm Z ddl m Z eej dƒZ Gdd „d ejƒZGd d „d eƒZdd d„Zddd„Zdd„Zddd„ZdS)éNé)ÚLOGGER)Ú check_version)Úbbox_iouÚprobiou)Úxywhr2xyxyxyxyz1.10.0cs~eZdZdZd‡fdd„ Ze ¡d d „ƒZd d „Zd d„Z dd„Z dd„Z ddd„Z dd„Z eddd„ƒZedd„ƒZ‡ZS) ÚTaskAlignedAssignera A task-aligned assigner for object detection. This class assigns ground-truth (gt) objects to anchors based on the task-aligned metric, which combines both classification and localization information. Attributes: topk (int): The number of top candidates to consider. num_classes (int): The number of object classes. bg_idx (int): Background class index. alpha (float): The alpha parameter for the classification component of the task-aligned metric. beta (float): The beta parameter for the localization component of the task-aligned metric. eps (float): A small value to prevent division by zero. é éPçð?ç@ç•Ö&è .>cs2tƒ ¡||_||_||_||_||_||_dS)zJInitialize a TaskAlignedAssigner object with customizable hyperparameters.N)ÚsuperÚ__init__ÚtopkÚ num_classesÚbg_idxÚalphaÚbetaÚeps)Úselfrrrrr©Ú __class__©úI/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/utils/tal.pyrs  zTaskAlignedAssigner.__init__c sÖ|jd|_|jd|_|j‰|jdkr2t |d|j¡t |¡t |¡t |d¡t |d¡fSz | ||||||¡WStj j yjt   d¡dd„||||||fDƒ}|j|Ž}t ‡fdd„|DƒƒYSw) a Compute the task-aligned assignment. Args: pd_scores (torch.Tensor): Predicted classification scores with shape (bs, num_total_anchors, num_classes). pd_bboxes (torch.Tensor): Predicted bounding boxes with shape (bs, num_total_anchors, 4). anc_points (torch.Tensor): Anchor points with shape (num_total_anchors, 2). gt_labels (torch.Tensor): Ground truth labels with shape (bs, n_max_boxes, 1). gt_bboxes (torch.Tensor): Ground truth boxes with shape (bs, n_max_boxes, 4). mask_gt (torch.Tensor): Mask for valid ground truth boxes with shape (bs, n_max_boxes, 1). Returns: target_labels (torch.Tensor): Target labels with shape (bs, num_total_anchors). target_bboxes (torch.Tensor): Target bounding boxes with shape (bs, num_total_anchors, 4). target_scores (torch.Tensor): Target scores with shape (bs, num_total_anchors, num_classes). fg_mask (torch.Tensor): Foreground mask with shape (bs, num_total_anchors). target_gt_idx (torch.Tensor): Target ground truth indices with shape (bs, num_total_anchors). References: https://github.com/Nioolek/PPYOLOE_pytorch/blob/master/ppyoloe/assigner/tal_assigner.py rr).rz7CUDA OutOfMemoryError in TaskAlignedAssigner, using CPUcSsg|]}| ¡‘qSr)Úcpu©Ú.0ÚtrrrÚ Qsz/TaskAlignedAssigner.forward..c3s|]}| ˆ¡VqdS)N)Útor©ÚdevicerrÚ Ss€z.TaskAlignedAssigner.forward..)ÚshapeÚbsÚ n_max_boxesr"ÚtorchÚ full_likerÚ zeros_likeÚ_forwardÚcudaÚOutOfMemoryErrorrÚwarningÚtuple) rÚ pd_scoresÚ pd_bboxesÚ anc_pointsÚ gt_labelsÚ gt_bboxesÚmask_gtÚ cpu_tensorsÚresultrr!rÚforward(s$     û  ûzTaskAlignedAssigner.forwardcCs¦| ||||||¡\}}} | || |j¡\} } }| ||| | ¡\} } }||9}|jddd}| |jddd}||||j d¡ d¡}||}| | ||  ¡| fS)a– Compute the task-aligned assignment. Args: pd_scores (torch.Tensor): Predicted classification scores with shape (bs, num_total_anchors, num_classes). pd_bboxes (torch.Tensor): Predicted bounding boxes with shape (bs, num_total_anchors, 4). anc_points (torch.Tensor): Anchor points with shape (num_total_anchors, 2). gt_labels (torch.Tensor): Ground truth labels with shape (bs, n_max_boxes, 1). gt_bboxes (torch.Tensor): Ground truth boxes with shape (bs, n_max_boxes, 4). mask_gt (torch.Tensor): Mask for valid ground truth boxes with shape (bs, n_max_boxes, 1). Returns: target_labels (torch.Tensor): Target labels with shape (bs, num_total_anchors). target_bboxes (torch.Tensor): Target bounding boxes with shape (bs, num_total_anchors, 4). target_scores (torch.Tensor): Target scores with shape (bs, num_total_anchors, num_classes). fg_mask (torch.Tensor): Foreground mask with shape (bs, num_total_anchors). target_gt_idx (torch.Tensor): Target ground truth indices with shape (bs, num_total_anchors). éÿÿÿÿT)ÚdimÚkeepdiméþÿÿÿ)Ú get_pos_maskÚselect_highest_overlapsr&Ú get_targetsÚamaxrÚ unsqueezeÚbool)rr/r0r1r2r3r4Úmask_posÚ align_metricÚoverlapsÚ target_gt_idxÚfg_maskÚ target_labelsÚ target_bboxesÚ target_scoresÚpos_align_metricsÚ pos_overlapsÚnorm_align_metricrrrr*Us  ÿzTaskAlignedAssigner._forwardc CsZ| ||¡}| ||||||¡\}} |j|| dd|j¡ ¡d} | ||} | || fS)aÝ Get positive mask for each ground truth box. Args: pd_scores (torch.Tensor): Predicted classification scores with shape (bs, num_total_anchors, num_classes). pd_bboxes (torch.Tensor): Predicted bounding boxes with shape (bs, num_total_anchors, 4). gt_labels (torch.Tensor): Ground truth labels with shape (bs, n_max_boxes, 1). gt_bboxes (torch.Tensor): Ground truth boxes with shape (bs, n_max_boxes, 4). anc_points (torch.Tensor): Anchor points with shape (num_total_anchors, 2). mask_gt (torch.Tensor): Mask for valid ground truth boxes with shape (bs, n_max_boxes, 1). Returns: mask_pos (torch.Tensor): Positive mask with shape (bs, max_num_obj, h*w). align_metric (torch.Tensor): Alignment metric with shape (bs, max_num_obj, h*w). overlaps (torch.Tensor): Overlaps between predicted and ground truth boxes with shape (bs, max_num_obj, h*w). r8)Ú topk_mask)Úselect_candidates_in_gtsÚget_box_metricsÚselect_topk_candidatesÚexpandrrA) rr/r0r2r3r1r4Ú mask_in_gtsrCrDÚ mask_topkrBrrrr<zs   z TaskAlignedAssigner.get_pos_maskc Cs"|jd}| ¡}tj|j|j|g|j|jd}tj|j|j|g|j|jd}tjd|j|jgtjd} tj |jd  dd¡  d|j¡| d<|  d¡| d<|| dd d …| df|||<|  d¡  d|jdd¡|} |  d¡  dd|d¡|} | | | ¡||<| |j¡| |j¡} | |fS) a8 Compute alignment metric given predicted and ground truth bounding boxes. Args: pd_scores (torch.Tensor): Predicted classification scores with shape (bs, num_total_anchors, num_classes). pd_bboxes (torch.Tensor): Predicted bounding boxes with shape (bs, num_total_anchors, 4). gt_labels (torch.Tensor): Ground truth labels with shape (bs, n_max_boxes, 1). gt_bboxes (torch.Tensor): Ground truth boxes with shape (bs, n_max_boxes, 4). mask_gt (torch.Tensor): Mask for valid ground truth boxes with shape (bs, n_max_boxes, h*w). Returns: align_metric (torch.Tensor): Alignment metric combining classification and localization. overlaps (torch.Tensor): IoU overlaps between predicted and ground truth boxes. r;©Údtyper"é)rU)Úendr8rrN)r$rAr'Úzerosr%r&rUr"ÚlongÚarangeÚviewrQÚsqueezer@Úiou_calculationÚpowrr) rr/r0r2r3r4ÚnarDÚ bbox_scoresÚindÚpd_boxesÚgt_boxesrCrrrrO•s $"z#TaskAlignedAssigner.get_box_metricscCst||ddd d¡ d¡S)a Calculate IoU for horizontal bounding boxes. Args: gt_bboxes (torch.Tensor): Ground truth boxes. pd_bboxes (torch.Tensor): Predicted boxes. Returns: (torch.Tensor): IoU values between each pair of boxes. FT)ÚxywhÚCIoUr8r)rr\Úclamp_©rr3r0rrrr]·s z#TaskAlignedAssigner.iou_calculationTNc Csàtj||jd|d\}}|dur|jdddd|jk |¡}| |d¡tj|jtj|j d}tj |dd…dd…dd…ftj|j d}t |jƒD]}|  d|dd…dd…||d…f|¡qK| |dkd¡|  |j¡S) az Select the top-k candidates based on the given metrics. Args: metrics (torch.Tensor): A tensor of shape (b, max_num_obj, h*w), where b is the batch size, max_num_obj is the maximum number of objects, and h*w represents the total number of anchor points. largest (bool): If True, select the largest values; otherwise, select the smallest values. topk_mask (torch.Tensor): An optional boolean tensor of shape (b, max_num_obj, topk), where topk is the number of top candidates to consider. If not provided, the top-k values are automatically computed based on the given metrics. Returns: (torch.Tensor): A tensor of shape (b, max_num_obj, h*w) containing the selected top-k candidates. r8)r9ÚlargestNT)r:rrTr)r'rÚmaxrÚ expand_asÚ masked_fill_rXr$Úint8r"Ú ones_likeÚrangeÚ scatter_add_r rU) rÚmetricsrhrMÚ topk_metricsÚ topk_idxsÚ count_tensorÚonesÚkrrrrPÄs** z*TaskAlignedAssigner.select_topk_candidatesc CsÒtj|jtj|jdd}|||j}| ¡ ¡|}| d|j d¡|}|  d¡tj |j d|j d|j ftj|jd}|  d| d¡d¡|dd…dd…df dd|j ¡} t | dk|d¡}|||fS) a Compute target labels, target bounding boxes, and target scores for the positive anchor points. Args: gt_labels (torch.Tensor): Ground truth labels of shape (b, max_num_obj, 1), where b is the batch size and max_num_obj is the maximum number of objects. gt_bboxes (torch.Tensor): Ground truth bounding boxes of shape (b, max_num_obj, 4). target_gt_idx (torch.Tensor): Indices of the assigned ground truth objects for positive anchor points, with shape (b, h*w), where h*w is the total number of anchor points. fg_mask (torch.Tensor): A boolean tensor of shape (b, h*w) indicating the positive (foreground) anchor points. Returns: target_labels (torch.Tensor): Shape (b, h*w), containing the target labels for positive anchor points. target_bboxes (torch.Tensor): Shape (b, h*w, 4), containing the target bounding boxes for positive anchor points. target_scores (torch.Tensor): Shape (b, h*w, num_classes), containing the target scores for positive anchor points. )rWrUr").Nr8rrrTrVN)r'rZr%Úint64r"r&rYÚflattenr[r$rfrXrÚscatter_r@ÚrepeatÚwhere) rr2r3rErFÚ batch_indrGrHrIÚfg_scores_maskrrrr>æs ý" zTaskAlignedAssigner.get_targetsc Csn|jd}|j\}}}| ddd¡ dd¡\}}tj|d|||dfdd |||d¡} |  d¡ |¡S) aw Select positive anchor centers within ground truth bounding boxes. Args: xy_centers (torch.Tensor): Anchor center coordinates, shape (h*w, 2). gt_bboxes (torch.Tensor): Ground truth bounding boxes, shape (b, n_boxes, 4). eps (float, optional): Small value for numerical stability. Defaults to 1e-9. Returns: (torch.Tensor): Boolean mask of positive anchors, shape (b, n_boxes, h*w). Note: b: batch size, n_boxes: number of ground truth boxes, h: height, w: width. Bounding box format: [x_min, y_min, x_max, y_max]. rr8rérVN©r9é)r$r[Úchunkr'ÚcatÚaminÚgt_) Ú xy_centersr3rÚ n_anchorsr%Ún_boxesÚ_ÚltÚrbÚ bbox_deltasrrrrNs  .z,TaskAlignedAssigner.select_candidates_in_gtscCs’| d¡}| ¡dkr?| d¡dk d|d¡}| d¡}tj|j|j|j d}|  d| d¡d¡t  |||¡  ¡}| d¡}| d¡}|||fS)aa Select anchor boxes with highest IoU when assigned to multiple ground truths. Args: mask_pos (torch.Tensor): Positive mask, shape (b, n_max_boxes, h*w). overlaps (torch.Tensor): IoU overlaps, shape (b, n_max_boxes, h*w). n_max_boxes (int): Maximum number of ground truth boxes. Returns: target_gt_idx (torch.Tensor): Indices of assigned ground truths, shape (b, h*w). fg_mask (torch.Tensor): Foreground mask, shape (b, h*w). mask_pos (torch.Tensor): Updated positive mask, shape (b, n_max_boxes, h*w). r;rr8rT) Úsumrir@rQÚargmaxr'rXr$rUr"rxrzÚfloat)rBrDr&rFÚmask_multi_gtsÚmax_overlaps_idxÚis_max_overlapsrErrrr=*s      z+TaskAlignedAssigner.select_highest_overlaps)r r r r r )TN)r )Ú__name__Ú __module__Ú __qualname__Ú__doc__rr'Úno_gradr7r*r<rOr]rPr>Ú staticmethodrNr=Ú __classcell__rrrrrs ,%" "- rc@s$eZdZdZdd„Zedd„ƒZdS)ÚRotatedTaskAlignedAssignerzSAssigns ground-truth objects to rotated bounding boxes using a task-aligned metric.cCst||ƒ d¡ d¡S)z)Calculate IoU for rotated bounding boxes.r8r)rr\rfrgrrrr]Lsz*RotatedTaskAlignedAssigner.iou_calculationcCs–t|ƒ}|jddd\}}}}||}||}||} ||jdd} ||jdd} | |jdd} | |jdd} | dk| | k@| dk@| | k@S)a… Select the positive anchor center in gt for rotated bounding boxes. Args: xy_centers (torch.Tensor): Anchor center coordinates with shape (h*w, 2). gt_bboxes (torch.Tensor): Ground truth bounding boxes with shape (b, n_boxes, 5). Returns: (torch.Tensor): Boolean mask of positive anchors with shape (b, n_boxes, h*w). rr;r~r8r)rÚsplitr‹)r„r3ÚcornersÚaÚbr‡ÚdÚabÚadÚapÚnorm_abÚnorm_adÚ ap_dot_abÚ ap_dot_adrrrrNPs  z3RotatedTaskAlignedAssigner.select_candidates_in_gtsN)r‘r’r“r”r]r–rNrrrrr˜Is r˜çà?c Csgg}}|dus J‚|dj|dj}}t|ƒD]i\}}t|tƒr,||jdd…nt||dƒt||dƒf\} } tj| ||d|} tj| ||d|} t r\tj | | ddnt  | | ¡\} } |  t  | | fd¡  dd¡¡|  tj| | df|||d ¡qt |¡t |¡fS) zGenerate anchors from features.NrrVr)rWr"rUÚij)Úindexingr8rT)rUr"Ú enumerateÚ isinstanceÚlistr$Úintr'rZÚ TORCH_1_10ÚmeshgridÚappendÚstackr[Úfullr) ÚfeatsÚstridesÚgrid_cell_offsetÚ anchor_pointsÚ stride_tensorrUr"ÚiÚstrideÚhÚwÚsxÚsyrrrÚ make_anchorsls  @$"r¼Tr8c CsX| d|¡\}}||}||}|r$||d}||} t || f|¡St ||f|¡S)z.Transform distance(ltrb) to box(xywh or xyxy).rV)r€r'r) Údistancer´rdr9rˆr‰Úx1y1Úx2y2Úc_xyÚwhrrrÚ dist2bbox{s rÂcCs4| dd¡\}}t ||||fd¡ d|d¡S)z#Transform bbox(xyxy) to dist(ltrb).rVr8rg{®Gáz„?)r€r'rrf)r´ÚbboxÚreg_maxr¾r¿rrrÚ bbox2dist‡s$rÅc Cs|jd|d\}}t |¡t |¡}}||djd|d\}} ||| |||| |} } tj| | g|d|} tj| ||g|dS)a Decode predicted rotated bounding box coordinates from anchor points and distribution. Args: pred_dist (torch.Tensor): Predicted rotated distance with shape (bs, h*w, 4). pred_angle (torch.Tensor): Predicted angle with shape (bs, h*w, 1). anchor_points (torch.Tensor): Anchor points with shape (h*w, 2). dim (int, optional): Dimension along which to split. Defaults to -1. Returns: (torch.Tensor): Predicted rotated bounding boxes with shape (bs, h*w, 4). rVr~r)r™r'ÚcosÚsinr) Ú pred_distÚ pred_angler´r9rˆr‰rÆrÇÚxfÚyfÚxÚyÚxyrrrÚ dist2rboxs  "rÏ)r¥)Tr8)r8)r'Útorch.nnÚnnÚrÚchecksrrprrÚopsrÚ __version__r¬ÚModulerr˜r¼rÂrÅrÏrrrrÚs     = #