o W h-5@spddlZddlmZddlmmZddlmZddlm Z ddl m Z m Z Gdddej Z d d d ZdS)Nlinear_sum_assignment)bbox_iou) xywh2xyxy xyxy2xywhcs,eZdZdZd fdd Zd d d ZZS)HungarianMatchera= A module implementing the HungarianMatcher, which is a differentiable module to solve the assignment problem in an end-to-end fashion. HungarianMatcher performs optimal assignment over the predicted and ground truth bounding boxes using a cost function that considers classification scores, bounding box coordinates, and optionally, mask predictions. Attributes: cost_gain (dict): Dictionary of cost coefficients: 'class', 'bbox', 'giou', 'mask', and 'dice'. use_fl (bool): Indicates whether to use Focal Loss for the classification cost calculation. with_mask (bool): Indicates whether the model makes mask predictions. num_sample_points (int): The number of sample points used in mask cost calculation. alpha (float): The alpha factor in Focal Loss calculation. gamma (float): The gamma factor in Focal Loss calculation. Methods: forward: Computes the assignment between predictions and ground truths for a batch. _cost_mask: Computes the mask cost and dice cost if masks are predicted. NTF1?@csJt|durdddddd}||_||_||_||_||_||_dS)a Initialize a HungarianMatcher module for optimal assignment of predicted and ground truth bounding boxes. The HungarianMatcher uses a cost function that considers classification scores, bounding box coordinates, and optionally mask predictions to perform optimal bipartite matching between predictions and ground truths. Args: cost_gain (dict, optional): Dictionary of cost coefficients for different components of the matching cost. Should contain keys 'class', 'bbox', 'giou', 'mask', and 'dice'. use_fl (bool, optional): Whether to use Focal Loss for the classification cost calculation. with_mask (bool, optional): Whether the model makes mask predictions. num_sample_points (int, optional): Number of sample points used in mask cost calculation. alpha (float, optional): Alpha factor in Focal Loss calculation. gamma (float, optional): Gamma factor in Focal Loss calculation. N)classbboxgioumaskdice)super__init__ cost_gainuse_fl with_masknum_sample_pointsalphagamma)selfrrrrrr __class__P/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/models/utils/ops.pyr!s  zHungarianMatcher.__init__cs|j\}} } tdkrddt|DS|d| }|jr%t|ntj|dd}|dd}|dd|f}|jrhd|j ||j d|d  } |j d||j |d  } | | } n| } | d| d d}d t| d| dd d d d}|jd | |jd||jd|}|jr|||||7}d|||B<||| d}ddt|dD}tdgdddfddt|DS)a Forward pass for HungarianMatcher. Computes costs based on prediction and ground truth and finds the optimal matching between predictions and ground truth based on these costs. Args: pred_bboxes (torch.Tensor): Predicted bounding boxes with shape (batch_size, num_queries, 4). pred_scores (torch.Tensor): Predicted scores with shape (batch_size, num_queries, num_classes). gt_cls (torch.Tensor): Ground truth classes with shape (num_gts, ). gt_bboxes (torch.Tensor): Ground truth bounding boxes with shape (num_gts, 4). gt_groups (List[int]): List of length equal to batch size, containing the number of ground truths for each image. masks (torch.Tensor, optional): Predicted masks with shape (batch_size, num_queries, height, width). gt_mask (List[torch.Tensor], optional): List of ground truth masks, each with shape (num_masks, Height, Width). Returns: (List[Tuple[torch.Tensor, torch.Tensor]]): A list of size batch_size, each element is a tuple (index_i, index_j), where: - index_i is the tensor of indices of the selected predictions (in order) - index_j is the tensor of indices of the corresponding selected ground truth targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) rcSs,g|]}tjgtjdtjgtjdfqSdtypetorchtensorlong).0_rrr Ts,z,HungarianMatcher.forward..dimNr g:0yE>?T)xywhGIoUrrrcSsg|] \}}t||qSrr)r'icrrrr)zscs<g|]\}\}}tj|tjdtj|tjd|fqSr r#)r'kr2j gt_groupsrrr)|s &)shapesumrangedetachviewrFsigmoidsoftmaxrrlog unsqueezeabsrsqueezerr _cost_maskisnanisinfcpu enumeratesplitr$ as_tensorcumsum_)r pred_bboxes pred_scores gt_bboxesgt_clsr7masksgt_maskbsnqncneg_cost_classpos_cost_class cost_class cost_bbox cost_giouCindicesrr6rforward;s:  &" &    zHungarianMatcher.forward)NTFrr r )NN)__name__ __module__ __qualname____doc__rr\ __classcell__rrrrr srd?r.Fc"s|r |dks |dur dS|d}t|} t|dkrdS|} | dkr(dn| } t|} |d} |d} |d}| d | }| d | d}|d | d }tj| | tj| jd | | }|dkrt |j |d k}t | d }tj |d||j|jd }|||<|dkrt|}|d d dfd dd |}t |dd dd}t|}||d7<||9}|||7}|jdddt|}tj|dd}td | }||}tj| ||j d | jd}tj| |d| jd}tdd|Dtjfddt| Ddd}tfddtd | D|||f<|||f<||}tj||gtjd}d||dd|f<t| D]u} | dkrmd|d | d | dd | d|f<| | dkrd|d | d | dd| d f<qLd|d | d | dd | d|f<d|d | d | ddd | f<qLdd|jt|ddD| ||gd}!||j||j||j|!fS)a Get contrastive denoising training group with positive and negative samples from ground truths. Args: batch (dict): A dict that includes 'gt_cls' (torch.Tensor with shape (num_gts, )), 'gt_bboxes' (torch.Tensor with shape (num_gts, 4)), 'gt_groups' (List[int]) which is a list of batch size length indicating the number of gts of each image. num_classes (int): Number of classes. num_queries (int): Number of queries. class_embed (torch.Tensor): Embedding weights to map class labels to embedding space. num_dn (int, optional): Number of denoising queries. cls_noise_ratio (float, optional): Noise ratio for class labels. box_noise_scale (float, optional): Noise scale for bounding box coordinates. training (bool, optional): If it's in training mode. Returns: padding_cls (Optional[torch.Tensor]): The modified class embeddings for denoising. padding_bbox (Optional[torch.Tensor]): The modified bounding boxes for denoising. attn_mask (Optional[torch.Tensor]): The attention mask for denoising. dn_meta (Optional[Dict]): Meta information for denoising. rN)NNNNr7r clsbboxes batch_idxr r*)r"devicerc.r r.r1)minmaxgư>)eps)rgr-cSs g|] }tjt|tjdqSr )r$r%r:r&)r'numrrrr)s z!get_cdn_group..cg|]}|qSrrr'r2 map_indicesmax_numsrrr)r+crlrrrmrnrrr)rqr!TcSsg|]}|dqS)r*)reshape)r'prrrr)s) dn_pos_idx dn_num_group dn_num_split)r9rilenrepeatr<r$aranger&rgrandr8nonzerorC randint_liker"r rand_likeclip_rlogitintzeroscatstackr:boolrGrIlistto)"batch num_classes num_queries class_embednum_dncls_noise_ratiobox_noise_scaletrainingr7 total_num num_grouprRrOgt_bboxb_idxdn_clsdn_bboxdn_b_idxneg_idxridx new_label known_bboxdiff rand_sign rand_part dn_cls_embed padding_cls padding_bboxpos_idxtgt_size attn_maskr2dn_metarrnr get_cdn_groupsz    ""    4444   r)rbrcr.F)r$torch.nnnntorch.nn.functional functionalr=scipy.optimizerultralytics.utils.metricsrultralytics.utils.opsrrModulerrrrrrs