o W hR@sZddlZddlZddlmZddlmZddlmZGdddeZ Gddde eZ dS) N)LoadVisualPrompt)DetectionPredictor)SegmentationPredictorcsXeZdZdZdfdd ZddZfddZdfd d Zfd d ZddZ Z S)YOLOEVPDetectPredictoraF A mixin class for YOLO-EVP (Enhanced Visual Prompting) predictors. This mixin provides common functionality for YOLO models that use visual prompting, including model setup, prompt handling, and preprocessing transformations. Attributes: model (torch.nn.Module): The YOLO model for inference. device (torch.device): Device to run the model on (CPU or CUDA). prompts (dict): Visual prompts containing class indices and bounding boxes or masks. Methods: setup_model: Initialize the YOLO model and set it to evaluation mode. set_return_vpe: Set whether to return visual prompt embeddings. set_prompts: Set the visual prompts for the model. pre_transform: Preprocess images and prompts before inference. inference: Run inference with visual prompts. Tcstj||dd|_dS)z Sets up the model for prediction. Args: model (torch.nn.Module): Model to load or use. verbose (bool): If True, provides detailed logging. )verboseTN)super setup_model done_warmup)selfmodelr __class__Y/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/models/yolo/yoloe/predict.pyr s z"YOLOEVPDetectPredictor.setup_modelcCs ||_dS)z Set the visual prompts for the model. Args: prompts (dict): Dictionary containing class indices and bounding boxes or masks. Must include a 'cls' key with class indices. N)prompts)r rrrr set_prompts+s z"YOLOEVPDetectPredictor.set_promptscstjddjdd}jdtdkrBdjdddjdd|}|dj _SdusNJdd t t r\t d d DsdJd d t t rrt d d DszJd d ttkrtksnJdtdtdtd fddt tD}tjjjj|ddj _S)a Preprocess images and prompts before inference. This method applies letterboxing to the input image and transforms the visual prompts (bounding boxes or masks) accordingly. Args: im (list): List containing a single input image. Returns: (list): Preprocessed image ready for model inference. Raises: ValueError: If neither valid bounding boxes nor masks are provided in the prompts. bboxesNmasksclsrzExpected bboxes, but got !cs|] }t|tjVqdSN isinstancenpndarray.0brrr Pz7YOLOEVPDetectPredictor.pre_transform..z#Expected List[np.ndarray], but got csrrrrrrrr!Sr"z-Expected same length for all inputs, but got vsc s@g|]}|jdd|jdd||qS)Nr)_process_single_imageshape)rircategoryimimgr rr Ys2z8YOLOEVPDetectPredictor.pre_transform..T) batch_first)r pre_transformrpoplenr$r% unsqueezetodevicerlistallrangetorchnnutilsrnn pad_sequence)r r)rvisualsr r'rr-5s.   .  $" z$YOLOEVPDetectPredictor.pre_transformNcs|durit|ritj|tjd}|jdkr|dddf}t|d|d|d|d}||9}|ddddft|d|d|dd7<|ddddft|d|d|dd7<n|durt|}t |}d||dk<nt d t ||||S) a Processes a single image by resizing bounding boxes or masks and generating visuals. Args: dst_shape (tuple): The target shape (height, width) of the image. src_shape (tuple): The original shape (height, width) of the image. category (str): The category of the image for visual prompts. bboxes (list | np.ndarray, optional): A list of bounding boxes in the format [x1, y1, x2, y2]. Defaults to None. masks (np.ndarray, optional): A list of masks corresponding to the image. Defaults to None. Returns: visuals: The processed visuals for the image. Raises: ValueError: If neither `bboxes` nor `masks` are provided. N)dtyperr.rg?rz$Please provide valid bboxes or masks) r/rarrayfloat32ndimminroundrr-stack ValueErrorr get_visuals)r dst_shape src_shaper(rrgain resized_masksr rrr$as "68  z,YOLOEVPDetectPredictor._process_single_imagecs tj|g|Rd|ji|S)a/ Run inference with visual prompts. Args: im (torch.Tensor): Input image tensor. *args (Any): Variable length argument list. **kwargs (Any): Arbitrary keyword arguments. Returns: (torch.Tensor): Model prediction results. vpe)r inferencer)r r)argskwargsr rrrKs z YOLOEVPDetectPredictor.inferencecCsT||t|jdksJd|jD]\}}}||}|j||jddSdS)a Processes the source to get the visual prompt embeddings (VPE). Args: source (str | Path | int | PIL.Image | np.ndarray | torch.Tensor | List | Tuple): The source of the image to make predictions on. Accepts various types including file paths, URLs, PIL images, numpy arrays, and torch tensors. Returns: (torch.Tensor): The visual prompt embeddings (VPE) from the model. rz get_vpe only supports one image!T)rJ return_vpeN) setup_sourcer/dataset preprocessr r)r source_im0sr)rrrget_vpes  zYOLOEVPDetectPredictor.get_vpe)T)NN) __name__ __module__ __qualname____doc__rrr-r$rKrU __classcell__rrr rr s , %rc@seZdZdZdS)YOLOEVPSegPredictorz$Predictor for YOLOE VP segmentation.N)rVrWrXrYrrrrr[sr[) numpyrr6ultralytics.data.augmentrultralytics.models.yolo.detectrultralytics.models.yolo.segmentrrr[rrrrs