o W h @sddlmZddlZddlmZddlmZmZm Z ddl m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZmZGd ddeZGdddeeZdS))deepcopyN) functional)YOLOConcatDatasetbuild_dataloaderbuild_yolo_dataset)LoadVisualPrompt)check_det_dataset)DetectionValidator)SegmentationValidator) YOLOEDetect) YOLOEModel)LOGGERTQDM) select_devicesmart_inference_modecsJeZdZdZeddZfddZddZed fd d ZZ S) YOLOEDetectValidatoraS A mixin class for YOLOE model validation that handles both text and visual prompt embeddings. This mixin provides functionality to validate YOLOE models using either text or visual prompt embeddings. It includes methods for extracting visual prompt embeddings from samples, preprocessing batches, and running validation with different prompt types. Attributes: device (torch.device): The device on which validation is performed. args (namespace): Configuration arguments for validation. dataloader (DataLoader): DataLoader for validation data. c Cst|tsJddt|jjdD}tjt||j dj |j d}tt|}d}|D]}|d d tj}tj|t|d} || 7}q0| |j }t|t||d } | D]b}||}|j|d |d d } |d } t| jdD]D} |d| | k d tjjdd}tj| jd|j dd}||dt|<|D]}||| | ||kd||7<qq{q^tj||dkddd||dk<d||dk<|dS)aa Extract visual prompt embeddings from training samples. This function processes a dataloader to compute visual prompt embeddings for each class using a YOLOE model. It normalizes the embeddings and handles cases where no samples exist for a class. Args: dataloader (torch.utils.data.DataLoader): The dataloader providing training samples. model (YOLOEModel): The YOLOE model from which to extract visual prompt embeddings. Returns: (torch.Tensor): Visual prompt embeddings with shape (1, num_classes, embed_dim). cSg|] }|ddqS/rsplit.0namerU/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/models/yolo/yoloe/val.py 2z6YOLOEDetectValidator.get_visual_pe..names)devicez)Get visual prompt embeddings from samplescls) minlength)totaldescimgvisuals)visual batch_idxrT)sortedN)dimp) isinstancer listdatasetdatavaluestorchzeroslenmodelembedr squeezetointuniquebincountr preprocess get_visual_perangeshapeonessumF normalize unsqueeze)self dataloaderr6r visual_pecls_visual_numr$batchr!countpbarpredsr(ipad_clscrrrr>!s4   &,   z"YOLOEDetectValidator.get_visual_pecs0t|}d|vr|d|dj|d<|S)zIPreprocess batch data, ensuring visuals are on the same device as images.r&r%)superr=r9r )rFrJ __class__rrr=Ps zYOLOEDetectValidator.preprocesscCszt|j||jj|d|jj|ddd}t|tr)|jD] }|j t qn|j t t ||jj|jj dddS)a Create a dataloader for LVIS training visual prompt samples. This function prepares a dataloader for visual prompt embeddings (VPE) using the LVIS dataset. It applies necessary transformations and configurations to the dataset and returns a dataloader for validation purposes. Args: data (dict): Dataset configuration dictionary containing paths and settings. Returns: (torch.utils.data.DataLoader): The dataLoader for visual prompt samples. valF)moderectr)shufflerank) rargsgetrrJr.rdatasets transformsappendrrworkers)rFr1r0drrrget_vpe_dataloaderWs(  z'YOLOEDetectValidator.get_vpe_dataloaderNFc s|durN|j|_|jj}ddt|jjjdD}|r5tdd|j _ | |j|}| ||ntd| |}| ||t||}|S|durX|sXJdt|j j|_t|trrd d lm} | ||jd d }||jt|p|j j} d dt| dD}|rtdd|j _ || } | | |}| ||tjt|d}|St|jdtrt|jddrt||Std| |}| ||tjt|d}|S)a9 Run validation on the model using either text or visual prompt embeddings. This method validates the model using either text prompts or visual prompts, depending on the `load_vp` flag. It supports validation during training (using a trainer object) or standalone validation with a provided model. Args: trainer (object, optional): Trainer object containing the model and device. model (YOLOEModel, optional): Model to validate. Required if `trainer` is not provided. refer_data (str, optional): Path to reference data for visual prompts. load_vp (bool): Whether to load visual prompts. If False, text prompts are used. Returns: (dict): Validation statistics containing metrics computed during validation. NcSrrrrrrrrrz1YOLOEDetectValidator.__call__..rz!Validate using the visual prompt.FzValidate using the text prompt.z5Refer data is only used for visual prompt validation.r)attempt_load_weightsT)r inplacecSrrrrrrrrr)r6rlrpc)r emar/rGr0r1r2r inforYhalfr> set_classes get_text_perQ__call__rr.strultralytics.nn.tasksraevalr9rr`rr6r hasattr) rFtrainerr6 refer_dataload_vprvpetpestatsrar1rGrRrrrizsL                zYOLOEDetectValidator.__call__)NNNF) __name__ __module__ __qualname____doc__rr>r=r`ri __classcell__rrrRrrs  .#rc@seZdZdZdS)YOLOESegValidatorzRYOLOE segmentation validator that supports both text and visual prompt embeddings.N)rtrurvrwrrrrrysry)copyrr3torch.nnrrCultralytics.datarrrultralytics.data.augmentrultralytics.data.utilsrultralytics.models.yolo.detectr ultralytics.models.yolo.segmentr ultralytics.nn.modules.headr rkr ultralytics.utilsr rultralytics.utils.torch_utilsrrrryrrrrs        *