o W h@s&ddlZddlZddlZddlZddlmZddlmZddlZddl m Z ddl m Z ddl mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAmBZBmCZCmDZDmEZEddlFmGZGmHZHmIZImJZJmKZKmLZLddlMmNZNmOZOmPZPddlQmRZRmSZSmTZTmUZUmVZVmWZWdd lXmYZYdd lZm[Z[dd l\m]Z]m^Z^m_Z_m`Z`maZambZbmcZcmdZdzddleZeWn efydZeYnwGd d d ej jgZhGdddehZiGdddeiZjGdddeiZkGdddeiZlGdddehZmGdddeiZnGdddeiZoGdddeiZpGdddepekZqGd d!d!ej jrZsejtd8d"d#ZuGd$d%d%ZvGd&d'd'ejwZxd9d)d*Zyd:d,d-Zzd:d.d/Z{d;d0d1Z|d2d3Z}d4d5Z~d6d7ZdS)<N)deepcopy)Path)check_class_names)8AIFIC1C2C2PSAC3C3TRELAN1OBBPSASPPSPPELANSPPFA2C2fAConvADown Bottleneck BottleneckCSPC2fC2fAttnC2fCIBC2fPSAC3GhostC3k2C3xCBFuseCBLinearClassifyConcatConvConv2 ConvTransposeDetectDWConvDWConvTranspose2dFocusGhostBottleneck GhostConvHGBlockHGStemImagePoolingAttnIndexLRPCHeadPoseRepC3RepConv RepNCSPELAN4RepVGGDW ResNetLayer RTDETRDecoderSCDownSegment TorchVision WorldDetect YOLOEDetect YOLOESegment v10Detect)DEFAULT_CFG_DICTDEFAULT_CFG_KEYSLOGGERcolorstremojis yaml_load)check_requirements check_suffix check_yaml) E2EDetectLossv8ClassificationLossv8DetectionLoss v8OBBLoss v8PoseLossv8SegmentationLoss)make_divisible)feature_visualization)fuse_conv_and_bnfuse_deconv_and_bninitialize_weightsintersect_dicts model_info scale_imgsmart_inference_mode time_synccseZdZdZddZdddZd dd Zd d Zd d Zd!ddZ d"ddZ d#ddZ fddZ d!ddZ d$ddZddZZS)% BaseModelz]The BaseModel class serves as a base class for all the models in the Ultralytics YOLO family.cOs:t|tr|j|g|Ri|S|j|g|Ri|S)a0 Perform forward pass of the model for either training or inference. If x is a dict, calculates and returns the loss for training. Otherwise, returns predictions for inference. Args: x (torch.Tensor | dict): Input tensor for inference, or dict with image tensor and labels for training. *args (Any): Variable length argument list. **kwargs (Any): Arbitrary keyword arguments. Returns: (torch.Tensor): Loss if x is a dict (training), or network predictions (inference). ) isinstancedictlosspredict)selfxargskwargsr_H/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/nn/tasks.pyforwardhs zBaseModel.forwardFNcCs|r||S|||||S)a Perform a forward pass through the network. Args: x (torch.Tensor): The input tensor to the model. profile (bool): Print the computation time of each layer if True. visualize (bool): Save the feature maps of the model if True. augment (bool): Augment image during prediction. embed (list, optional): A list of feature vectors/embeddings to return. Returns: (torch.Tensor): The last output of the model. )_predict_augment _predict_once)r[r\profile visualizeaugmentembedr_r_r`rZzs zBaseModel.predictcsggg}}|jD]o}|jdkr(t|jtr|jn fdd|jD|r1|||||j|jvr?nd|rNt|j |j|d|rz|j|vrz|t j j ddd|jt|krzt jt |ddd Sq S) a Perform a forward pass through the network. Args: x (torch.Tensor): The input tensor to the model. profile (bool): Print the computation time of each layer if True. visualize (bool): Save the feature maps of the model if True. embed (list, optional): A list of feature vectors/embeddings to return. Returns: (torch.Tensor): The last output of the model. c g|] }|dkr n|qSrhr_.0jr\yr_r`  z+BaseModel._predict_once..Nsave_dirrururdimmodelfrWint_profile_one_layerappendisaverMtypetorchnn functionaladaptive_avg_pool2dsqueezemaxunbindcat)r[r\rdrergdt embeddingsmr_rnr`rcs   ,"zBaseModel._predict_oncecCst|jjd||S)zFPerform augmentations on input image x and return augmented inference.zR does not support 'augment=True' prediction. Reverting to single-scale prediction.)r?warning __class____name__rc)r[r\r_r_r`rbs  zBaseModel._predict_augmentc Cs||jdko t|t}tr#tj||r|n|gdddddnd}t}tdD] }||r5|n|q,|t|d||jdkr[t d d d d d d d d dt |ddd |dd |j dd|j |rt t |dd dd d dd ddSdS)a8 Profile the computation time and FLOPs of a single layer of the model on a given input. Args: m (torch.nn.Module): The layer to be profiled. x (torch.Tensor): The input data to the layer. dt (list): A list to store the computation time of the layer. rhF)inputsverbosergeA dz time (ms)z>10s GFLOPsparamsz modulez10.2f10.0f -z TotalN)ryrWlistthoprdcopyrUranger}r?infonprsum)r[rr\rcflopst_r_r_r`r|s 2 ".*zBaseModel._profile_one_layerTcCs|sw|jD]g}t|tttfr2t|dr2t|tr!|t |j |j |_ t |d|j |_t|trMt|drMt|j|j |_t |d|j |_t|trZ||j |_t|trg||j |_t|trp|q |j|d|S)z Fuse the `Conv2d()` and `BatchNorm2d()` layers of the model into a single layer for improved computation efficiency. Returns: (torch.nn.Module): The fused model is returned. bn)r)is_fusedrymodulesrWr!r"r%hasattr fuse_convsrNconvrdelattr forward_fuserar#rOconv_transposer1r3fuser<r)r[rrr_r_r`rs.       zBaseModel.fusercs8tddtjjDtfdd|D|kS)a? Check if the model has less than a certain threshold of BatchNorm layers. Args: thresh (int, optional): The threshold number of BatchNorm layers. Returns: (bool): True if the number of BatchNorm layers in the model is less than the threshold, False otherwise. css |] \}}d|vr|VqdS)NormNr_rlkvr_r_r` z%BaseModel.is_fused..c3s|]}t|VqdSN)rW)rlrrr_r`r)tuplerr__dict__itemsrr)r[threshr_rr`rs zBaseModel.is_fusedcCst||||dS)a) Print model information. Args: detailed (bool): If True, prints out detailed information about the model. verbose (bool): If True, prints out the model information. imgsz (int): The size of the image that the model will be trained on. )detailedrimgsz)rR)r[rrrr_r_r`rs zBaseModel.infocsHt|}|jd}t|tr"||j|_||j|_||j|_|S)a Apply a function to all tensors in the model that are not parameters or registered buffers. Args: fn (function): The function to apply to the model. Returns: (BaseModel): An updated BaseModel object. rh)super_applyryrWr$strideanchorsstrides)r[fnrrr_r`rs    zBaseModel._applycCspt|tr |dn|}|}t||}|j|dd|r6tdt|dt|j ddSdS)z Load weights into the model. Args: weights (dict | torch.nn.Module): The pre-trained weights to be loaded. verbose (bool, optional): Whether to log the transfer progress. ryF)strictz Transferred /z items from pretrained weightsN) rWrXfloat state_dictrQload_state_dictr?rlenry)r[weightsrrycsdr_r_r`loads *zBaseModel.loadcCs@t|dddur ||_|dur||dn|}|||S) Compute loss. Args: batch (dict): Batch to compute loss on. preds (torch.Tensor | List[torch.Tensor], optional): Predictions. criterionNimg)getattrinit_criterionrrar[batchpredsr_r_r`rY!s  zBaseModel.losscCstd)z0Initialize the loss criterion for the BaseModel.z4compute_loss() needs to be implemented by task heads)NotImplementedErrorr[r_r_r`r/zBaseModel.init_criterion)FFFN)FFNT)r)FTrr)r __module__ __qualname____doc__rarZrcrbr|rrrrrrYr __classcell__r_r_rr`rVes     rVcsLeZdZdZdfdd Zfdd Zedd d Zd dZddZ Z S)DetectionModelzYOLO detection model. yolo11n.yamlNTc stt|tr |nt|_jddddkr*tddjddd<|jd<|rK|jdkrKtd jdd ||jd<t t j||d \_ _ d d t jdD_jdd_tj ddd_j dttrdj_fdd}tfdd|td|D_j_ntdg_t|rňtddSdS)aZ Initialize the YOLO detection model with the given config and parameters. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. backbonerrSilencezYOLOv9 `Silence` module is deprecated in favor of torch.nn.Identity. Please delete local *.pt file and re-download the latest model checkpoint.z nn.IdentitychannelsncOverriding model.yaml nc= with nc=chrcSi|]}||qSr_r_rlr~r_r_r` Pz+DetectionModel.__init__..inplaceTrhend2endFcs>jr |dStttttfr|dS|S)z_Perform a forward pass through the model, handling different Detect subclass types accordingly.one2manyr)rrarWr7r;r/r )r\)rr[r_r`_forwardZs*z)DetectionModel.__init__.._forwardcsg|] }|jdqS))shaperlr\)sr_r`rp`z+DetectionModel.__init__..ru N)r__init__rWrXyaml_model_loadyamlr?rr parse_modelrryrrnamesgetrrrr$rtensorzerosr bias_initTensorrP)r[cfgrrrrr)rrr[r`r7s:     * zDetectionModel.__init__c st|dds |jjdkrtd||S|jdd}gd}gd}g}t||D],\}}t|r8| |n||t |j d }t |d } || |||} || q,||}t|d dfS) z Perform augmentations on input image x and return augmented inference and train outputs. Args: x (torch.Tensor): Input image tensor. Returns: (torch.Tensor): Augmented inference output. rFrzLModel does not support 'augment=True', reverting to single-scale prediction.rN)rug(\?gq= ףp?)NrN)gsrrh)rrrr?rrcrziprSflipr{rrrrZ _descale_predr}_clip_augmentedrr) r[r\img_sizerrzrosifixiyirr_r`rbls  &  zDetectionModel._predict_augmentruc Cs|ddddf|<|ddd|j|df|\}}}}|dkr,|d|}n |dkr6|d|}t||||f|S)a De-scale predictions following augmented inference (inverse operation). Args: p (torch.Tensor): Predictions tensor. flips (int): Flip type (0=none, 2=ud, 3=lr). scale (float): Scale factor. img_size (tuple): Original image size (height, width). dim (int): Dimension to split at. Returns: (torch.Tensor): De-scaled predictions. Nrurrr)splitrrr) pflipsscaler rwr\rowhclsr_r_r`rs& zDetectionModel._descale_predcs|jdjtddtD}d}|djd|tddt|D}|ddd| f|d<|djd|tfd dt|D}|dd|df|d<|S) z Clip YOLO augmented inference tails. Args: y (List[torch.Tensor]): List of detection tensors. Returns: (List[torch.Tensor]): Clipped detection tensors. rhcs|]}d|VqdSrNr_rr_r_r`rz1DetectionModel._clip_augmented..rurcsrrr_rr_r_r`rr.Nc3s |] }dd|VqdS)rruNr_rnlr_r`rr)ryrrrr)r[roger~r_rr`r s (,zDetectionModel._clip_augmentedcCst|ddr t|St|S)z5Initialize the loss criterion for the DetectionModel.rF)rrFrHrr_r_r`rszDetectionModel.init_criterion)rrNT)ru) rrrrrrb staticmethodrr rrr_r_rr`r4s 5 rc*eZdZdZd fdd Zdd ZZS) OBBModelz'YOLO Oriented Bounding Box (OBB) model.yolo11n-obb.yamlrNTctj||||ddS)aL Initialize YOLO OBB model with given config and parameters. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. rrrrNrrr[rrrrrr_r`r zOBBModel.__init__cCt|S)z,Initialize the loss criterion for the model.)rIrr_r_r`rrzOBBModel.init_criterion)r rNTrrrrrrrr_r_rr`r rcr) SegmentationModelzYOLO segmentation model.yolo11n-seg.yamlrNTcr!)aW Initialize YOLOv8 segmentation model with given config and parameters. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. r"Nr#r$rr_r`rr%zSegmentationModel.__init__cCr&)z8Initialize the loss criterion for the SegmentationModel.)rKrr_r_r`rrz SegmentationModel.init_criterion)r*rNTr'r_r_rr`r)r(r)cs*eZdZdZd fdd Zd d ZZS) PoseModelzYOLO pose model.yolo11n-pose.yamlrNNNTcsht|ts t|}t|r(t|t|dkr(td|dd|||d<tj||||ddS)ak Initialize YOLOv8 Pose model. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. data_kpt_shape (tuple): Shape of keypoints data. verbose (bool): Whether to display model information. kpt_shapez Overriding model.yaml kpt_shape=z with kpt_shape=r"N) rWrXranyrr?rrr)r[rrrdata_kpt_shaperrr_r`rs zPoseModel.__init__cCr&)z0Initialize the loss criterion for the PoseModel.)rJrr_r_r`rrzPoseModel.init_criterion)r,rNr-Tr'r_r_rr`r+sr+cs>eZdZdZdfdd Zdd Zed d Zd d ZZ S)ClassificationModelzYOLO classification model.yolo11n-cls.yamlrNTcst|||||dS)ae Initialize ClassificationModel with YAML, channels, number of classes, verbose flag. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. N)rr _from_yamlr$rr_r`rs zClassificationModel.__init__cCst|tr|nt||_|jd|}|jd<|r5||jdkr5td|jdd|||jd<n |sB|jddsBtdtt |j||d\|_ |_ t dg|_d d t|jdD|_|dS) aS Set YOLOv8 model configurations and define the model architecture. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. rrrrNzFnc not specified. Must specify nc in model.yaml or function arguments.rrucSrr_r_rr_r_r`rrz2ClassificationModel._from_yaml..)rWrXrrrr?r ValueErrorrrryrrrrrrr$r_r_r`r3s   zClassificationModel._from_yamlcCstt|dr |jn|d\}}t|tr+|jj|kr)tj |jj ||_dSdSt|tj j rG|j|krEt ||tj |j |dSdSt|tj j rdd|D}tj j |vrt|d|dddtj j }||j|krtj ||j |||<dSdStj j|vrt|d|dddtj j}||j|krtj j||j|||j||j||jdud||<dSdSdSdS)z Update a TorchVision classification model to class count 'n' if required. Args: model (torch.nn.Module): Model to update. nc (int): New number of classes. ryrhcSsg|]}t|qSr_)rrr_r_r`rp/rz7ClassificationModel.reshape_outputs..ruN)bias)rrrynamed_childrenrWrlinear out_featuresrrLinear in_featuressetattr SequentialrindexConv2d out_channels in_channels kernel_sizerr5)ryrnamertypesr~r_r_r`reshape_outputss2$    $ $&z#ClassificationModel.reshape_outputscCstS)z:Initialize the loss criterion for the ClassificationModel.)rGrr_r_r`r;sz"ClassificationModel.init_criterion)r2rNT) rrrrrr3rrDrrr_r_rr`r1s  r1cs>eZdZdZdfdd Zdd Zdd d Zdd dZZS)RTDETRDetectionModelaR RTDETR (Real-time DEtection and Tracking using Transformers) Detection Model class. This class is responsible for constructing the RTDETR architecture, defining loss functions, and facilitating both the training and inference processes. RTDETR is an object detection and tracking model that extends from the DetectionModel base class. Methods: init_criterion: Initializes the criterion used for loss calculation. loss: Computes and returns the loss during training. predict: Performs a forward pass through the network and returns the output. rtdetr-l.yamlrNTcr!)a7 Initialize the RTDETRDetectionModel. Args: cfg (str | dict): Configuration file name or path. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Print additional information during initialization. r"Nr#r$rr_r`rNr%zRTDETRDetectionModel.__init__cCsddlm}||jddS)z;Initialize the loss criterion for the RTDETRDetectionModel.r)RTDETRDetectionLossT)ruse_vfl)ultralytics.models.utils.lossrGr)r[rGr_r_r`rZs z#RTDETRDetectionModel.init_criterioncslt|ds ||_|d}t|}|dfddt|D}|dj|jtjd d|d j|jd j|jtjd d|d }|d urP|j ||d n|}|j rW|n|d\}}} } } | d urid\} } ntj || ddd\} }tj || ddd\} }t | d|g}t | d|g}|j||f|| | | dttjfdddD|jd fS)aK Compute the loss for the given batch of data. Args: batch (dict): Dictionary containing image and label data. preds (torch.Tensor, optional): Precomputed model predictions. Returns: (tuple): A tuple containing the total loss and main three losses in a tensor. rr batch_idxcsg|] }|kqSr_)ritemr)rJr_r`rprsz-RTDETRDetectionModel.loss..r)dtyperhbboxesdevice)rrMrJ gt_groupsN)rrur- dn_num_splitrrvr) dn_bboxes dn_scoresdn_metacsg|]}|qSr_)detach)rlr)rYr_r`rps) loss_giou loss_class loss_bbox)rrrrrtorOrlongviewrZtrainingrr unsqueezervalues as_tensor)r[rrrbsrPtargets dec_bboxes dec_scores enc_bboxes enc_scoresrTrRrSr_)rJrYr`rY`s2  zRTDETRDetectionModel.lossFc s&ggg}}|jddD]o} | jdkr,t| jtr!| jn fdd| jD|r5|| || | j|jvrCnd|rRt| j | j|d|r~| j|vr~|t j j ddd| jt|kr~t jt |ddd Sq|jd} | fd d| jD|S) aH Perform a forward pass through the model. Args: x (torch.Tensor): The input tensor. profile (bool): If True, profile the computation time for each layer. visualize (bool): If True, save feature maps for visualization. batch (dict, optional): Ground truth data for evaluation. augment (bool): If True, perform data augmentation during inference. embed (list, optional): A list of feature vectors/embeddings to return. Returns: (torch.Tensor): Model's output tensor. Nrhcrirjr_rkrnr_r`rprqz0RTDETRDetectionModel.predict..rrrtrurrvcg|]}|qSr_r_rk)ror_r`rprrx) r[r\rdrerrfrgrrrheadr_rnr`rZs$ ," zRTDETRDetectionModel.predict)rFrNTrFFNFN) rrrrrrrYrZrr_r_rr`rE@s   -rEcs@eZdZdZdfdd Zdd d Zdd d ZdddZZS) WorldModelzYOLOv8 World Model.yolov8s-world.yamlrNTcs2td|pdd|_d|_tj||||ddS)aP Initialize YOLOv8 world model with given config and parameters. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. ruPNr")rrandn txt_feats clip_modelrrr$rr_r`rs zWorldModel.__init__rkcszddl}Wntytdddl}Ynwt|dds(|r(|dd|_|r-|jn|ddtj}| | |}fdd| |D}t |dkrY|dnt j|dd }||jd d d d }|d t ||jd |_t ||jd _dS)a- Set classes in advance so that model could do offline-inference without clip model. Args: text (List[str]): List of class names. batch (int): Batch size for processing text tokens. cache_clip_model (bool): Whether to cache the CLIP model. rNz+git+https://github.com/ultralytics/CLIP.gitrozViT-B/32cg|] }|qSr_ encode_textrUrltokenryr_r`rprz*WorldModel.set_classes..rurvrrhT)rrwkeepdim)clip ImportErrorrCrrronext parametersrOtokenizerYrrrrnormreshaperrnryr)r[textrcache_clip_modelrwrO text_tokenrnr_rur` set_classess&    "zWorldModel.set_classesFc s|dur|jn|jjjd}t|tks|jdjr(|jddd}| }ggg}} |jD]} | j dkrTt | j t rI| j n fdd| j D|r]| | |t | trh| |nt | trs| |nt | tr~| |}n| | j|jvrnd|rt| j| j|d|r| j|vr| tjjddd| jt|krtjt| d dd Sq7S) a[ Perform a forward pass through the model. Args: x (torch.Tensor): The input tensor. profile (bool): If True, profile the computation time for each layer. visualize (bool): If True, save feature maps for visualization. txt_feats (torch.Tensor, optional): The text features, use it if it's given. augment (bool): If True, perform data augmentation during inference. embed (list, optional): A list of feature vectors/embeddings to return. Returns: (torch.Tensor): Model's output tensor. NrOrLrhrcrirjr_rkrnr_r`rprqz&WorldModel.predict..rrrtrurv)rnrYrOrLrryexportexpandrclonerzrWr{r|rr9r,r}r~rrMrrrrrrrrr) r[r\rdrernrfrg ori_txt_featsrrrr_rnr`rZs4   ,      "zWorldModel.predictcCs>t|ds ||_|dur|j|d|dd}|||S)rrNrrn)rn)rrrrarr_r_r`rYs   zWorldModel.loss)rjrNT)rkTrhr) rrrrrrrZrYrr_r_rr`ris   +ricsveZdZdZdfdd Zedd d Zed d ZddZddZ ddZ ddZ dddZ dddZ ZS) YOLOEModelzYOLOE detection model.yoloe-v8s.yamlrNTcr!)aI Initialize YOLOE model with given config and parameters. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. r"Nr#r$rr_r`rr%zYOLOEModel.__init__rkFc sddlm}t|jj}t|dds|r|d|d|_|r"|jn|d|d|}fdd| |D}t |d krC|dnt j |dd }| d t ||jd }|rZ|S|jr_J|jd } t| tskJ| |S) a Set classes in advance so that model could do offline-inference without clip model. Args: text (List[str]): List of class names. batch (int): Batch size for processing text tokens. cache_clip_model (bool): Whether to cache the CLIP model. without_reprta (bool): Whether to return text embeddings cooperated with reprta module. Returns: (torch.Tensor): Text positional embeddings. r)build_text_modelroNzmobileclip:bltrNcrpr_rqrsrur_r`rp=rz*YOLOEModel.get_text_pe..rurvrh)ultralytics.nn.text_modelrryryrzrOrror{rrrrr}rr\rWr:get_tpe) r[r~rrwithout_reprtarrOrrnrgr_rur` get_text_pe&s  "   zYOLOEModel.get_text_pecCs|||ddS)z Get visual embeddings. Args: img (torch.Tensor): Input image tensor. visual (torch.Tensor): Visual features. Returns: (torch.Tensor): Visual positional embeddings. T)vpe return_vper_)r[rvisualr_r_r` get_visual_peHs zYOLOEModel.get_visual_pecCs|jrJ|jd}t|tsJt|j}|tdd|j d|j d |t ddt t||j|jD|jd_t|j|jD]\}}t|t jsTJt|t js\J|d=|d=qHt||jd_t||_dS)z Set vocabulary for the prompt-free model. Args: vocab (nn.ModuleList): List of vocabulary items. names (List[str]): List of class names. rhrurrcss6|]\}\}}}t||d|d|dkdVqdS)rhr)enabledN)r.)rlr~rpflocr_r_r`rgs   z'YOLOEModel.set_vocab..N)r\ryrWr:ryrzrOremptyr]rYr ModuleList enumeratercv3cv2lrpcr<rrrr)r[vocabrrgrOloc_headcls_headr_r_r` set_vocabVs  & zYOLOEModel.set_vocabcCs|jrJ|jd}t|tsJ|jrJ||}|||t|jj }| |j |t }|jD]}t|t jsCJ||dq9|S)z Get fused vocabulary layer from the model. Args: names (list): List of class names. Returns: (nn.ModuleList): List of vocabulary modules. rh)r\ryrWr:rrrryrzrOrperYrrrr<r})r[rrgtperOrrr_r_r` get_vocabss     zYOLOEModel.get_vocabcCsJt|jddr Jd|jdksJ||_t||jd_t||_dS)z Set classes in advance so that model could do offline-inference without clip model. Args: names (List[str]): List of class names. embeddings (torch.Tensor): Embeddings tensor. rhrz^Prompt-free model does not support setting classes. Please try with Text/Visual prompt models.rN)rryndimrrrrr)r[rrr_r_r`rszYOLOEModel.set_classesc Csrg}|dur|jdks J|||dur"|jdksJ|||s2|t|dtdddtj|ddS)a Get class positional embeddings. Args: tpe (torch.Tensor, optional): Text positional embeddings. vpe (torch.Tensor, optional): Visual positional embeddings. Returns: (torch.Tensor): Class positional embeddings. Nrrrurkrlrv)rr}rrrr)r[rrall_per_r_r` get_cls_pes   zYOLOEModel.get_cls_pec sggg} } jd} |jD]} | jdkr-t| jtr"| jn fdd| jD|r6|| | t| tr|durE| |nd}|rX|dusOJ|jrTJ|S| | ||j dj dj d} | jd| ksv| jr}| | dd} | | n| | j|jvrnd|rt| j| j|d|r| j|vr| tjjddd| jt|krtjt| d dd SqS) a Perform a forward pass through the model. Args: x (torch.Tensor): The input tensor. profile (bool): If True, profile the computation time for each layer. visualize (bool): If True, save feature maps for visualization. tpe (torch.Tensor, optional): Text positional embeddings. augment (bool): If True, perform data augmentation during inference. embed (list, optional): A list of feature vectors/embeddings to return. vpe (torch.Tensor, optional): Visual positional embeddings. return_vpe (bool): If True, return visual positional embeddings. Returns: (torch.Tensor): Model's output tensor. rrhcrirjr_rkrnr_r`rprqz&YOLOEModel.predict..Nrrrrtrurv)rryrzrWr{r|r:get_vper\rrrYrOrLrrr}r~rrMrrrrrrrrr)r[r\rdrerrfrgrrrrbrcls_per_rnr`rZs6   ,   ( "zYOLOEModel.predictcCtt|dsddlm}|dddu}|r||n||_|dur4|j|d|dd|ddd}|||S) rrr) TVPDetectLossvisualsNrrnrr)rultralytics.utils.lossrrrrra)r[rrr visual_promptr_r_r`rY  $ zYOLOEModel.loss)rrNT)rkFF)FFNFNNFr)rrrrrrTrrrrrrrZrYrr_r_rr`rs !   0rcs,eZdZdZd fdd Zd dd ZZS) YOLOESegModelzYOLOE segmentation model.yoloe-v8s-seg.yamlrNTcr!)aV Initialize YOLOE segmentation model with given config and parameters. Args: cfg (str | dict): Model configuration file path or dictionary. ch (int): Number of input channels. nc (int, optional): Number of classes. verbose (bool): Whether to display model information. r"Nr#r$rr_r`rr%zYOLOESegModel.__init__cCr) rrr)TVPSegmentLossrNrrnr)rrrrrrra)r[rrrrr_r_r`rYrzYOLOESegModel.loss)rrNTr)rrrrrrYrr_r_rr`rs rcs*eZdZdZfddZdddZZS)EnsemblezEnsemble of models.cstdS)z!Initialize an ensemble of models.Nr#rrr_r`rszEnsemble.__init__Fcs,fdd|D}t|d}|dfS)a Generate the YOLO network's final layer. Args: x (torch.Tensor): Input tensor. augment (bool): Whether to augment the input. profile (bool): Whether to profile the model. visualize (bool): Whether to visualize the features. Returns: (tuple): Tuple containing the concatenated predictions and None. csg|] }|dqS)rr_)rlmodulerfrdrer\r_r`rp,z$Ensemble.forward..rN)rr)r[r\rfrdreror_rr`ras zEnsemble.forward)FFF)rrrrrrarr_r_rr`rs rc cs|duri}|dur i}ddl}ddlm}zJ|D]!\}}|dd\}}|dd\}} t|||t||| q|D] \}}|||j|<qBdVW|D] }||jvr_|j|=qTdS|D] }||jvro|j|=qdw)a Context manager for temporarily adding or modifying modules in Python's module cache (`sys.modules`). This function can be used to change the module paths during runtime. It's useful when refactoring code, where you've moved a module from one location to another, but you still want to support the old import paths for backwards compatibility. Args: modules (dict, optional): A dictionary mapping old module paths to new module paths. attributes (dict, optional): A dictionary mapping old module attributes to new module attributes. Examples: >>> with temporary_modules({"old.module": "new.module"}, {"old.module.attribute": "new.module.attribute"}): >>> import old.module # this will now import new.module >>> from old.module import attribute # this will now import new.module.attribute Note: The changes are only in effect inside the context manager and are undone once the context manager exits. Be aware that directly manipulating `sys.modules` can lead to unpredictable results, especially in larger applications or libraries. Use this function with caution. Nr) import_module.ru)sys importlibrrrsplitr;rr) r attributesrroldnew old_moduleold_attr new_modulenew_attrr_r_r`temporary_modules6s0   rc@s eZdZdZddZddZdS) SafeClasszAA placeholder class to replace unknown classes during unpickling.cOdS)z6Initialize SafeClass instance, ignoring all arguments.Nr_r[r]r^r_r_r`rjzSafeClass.__init__cOr)z/Run SafeClass instance, ignoring all arguments.Nr_rr_r_r`__call__nrzSafeClass.__call__N)rrrrrrr_r_r_r`rgs rcs eZdZdZfddZZS) SafeUnpicklerz>Custom Unpickler that replaces unknown classes with SafeClass.csd}||vr t||StS)z Attempt to find a class, returning SafeClass if not among safe modules. Args: module (str): Module name. name (str): Class name. Returns: (type): Found class or SafeClass. )r collectionszcollections.abcbuiltinsmathnumpy)r find_classr)r[rrB safe_modulesrr_r`rvs  zSafeUnpickler.find_class)rrrrrrr_r_rr`rssrFc Csrddlm}t|dd||}zStddddd d d d d ;|rLtd}t|_dd|_t |d}t j||d}Wdn1sFwYnt j|dd}Wdn1s]wYWn>t y}z2|j dkr{t td|d|t|d|j d|j dt|j t j|dd}WYd}~nd}~wwt|tstd|dd|ji}||fS)a Attempts to load a PyTorch model with the torch.load() function. If a ModuleNotFoundError is raised, it catches the error, logs a warning message, and attempts to install the missing module via the check_requirements() function. After installation, the function again attempts to load the model using torch.load(). Args: weight (str): The file path of the PyTorch model. safe_only (bool): If True, replace unknown classes with SafeClass during loading. Returns: ckpt (dict): The loaded model checkpoint. file (str): The loaded filename. Examples: >>> from ultralytics.nn.tasks import torch_safe_load >>> ckpt, file = torch_safe_load("path/to/best.pt", safe_only=True) r)attempt_download_assetz.pt)filesuffixzultralytics.utilszultralytics.models.yolozultralytics.data)zultralytics.yolo.utilszultralytics.yolo.v8zultralytics.yolo.dataztorch.nn.Identityz#ultralytics.nn.tasks.DetectionModelz$ultralytics.utils.loss.E2EDetectLoss)z$ultralytics.nn.modules.block.Silencez*ultralytics.nn.tasks.YOLOv10DetectionModelz$ultralytics.utils.loss.v10DetectLoss)rr safe_picklecSs t|Sr)rr)file_objr_r_r`s z!torch_safe_load..rb) pickle_moduleNcpu) map_locationmodelsu ERROR ❌️ aw appears to be an Ultralytics YOLOv5 model originally trained with https://github.com/ultralytics/yolov5. This model is NOT forwards compatible with YOLOv8 at https://github.com/ultralytics/ultralytics. Recommend fixes are to train a new model using the latest 'ultralytics' package or to run a command with an official Ultralytics model, i.e. 'yolo predict model=yolo11n.pt'z appears to require 'zK', which is not in Ultralytics requirements. AutoInstall will run now for 'z' but this feature will be removed in the future. Recommend fixes are to train a new model using the latest 'ultralytics' package or to run a command with an official Ultralytics model, i.e. 'yolo predict model=yolo11n.pt'z The file 'z' appears to be improperly saved or formatted. For optimal results, use model.save('filename.pt') to correctly save YOLO models.ry)ultralytics.utils.downloadsrrDrrC ModuleTyper UnpicklerropenrModuleNotFoundErrorrB TypeErrorrAr?rrCrWrXry)weight safe_onlyrrrrzckptrr_r_r`torch_safe_loadsd            rTc stt|tr |n|gD]O}t|\}}d|vr!it|dnd}|dp+|d|}||_||_ t ||_ t |dsHt dg|_|rWt |drW|n|q D]}t |drl||_qat|t jjr{t |d s{d|_qatd krd Std |d dD] } t| td| qtt t ddDj_tfddDsJdddDS)a Load an ensemble of models weights=[a,b,c] or a single model weights=[a] or weights=a. Args: weights (str | List[str]): Model weights path(s). device (torch.device, optional): Device to load model to. inplace (bool): Whether to do inplace operations. fuse (bool): Whether to fuse model. Returns: (torch.nn.Module): Loaded model. train_argsNemaryr@@rrrecompute_scale_factorrurhzEnsemble created with  )rrrrcSsg|]}|jqSr_)rrrlrr_r_r`rpsz(attempt_load_weights..c3s |] }dj|jkVqdS)rNrrensembler_r`r rz'attempt_load_weights..zModels differ in class counts cSsg|]}|jqSr_rrr_r_r`rp s) rrWrrr=rrYrr]pt_pathguess_model_tasktaskrrrrr}revalrrrUpsamplerrr?rr;rr{argmaxall) rrOrrwrr]ryrrr_rr`attempt_load_weightss2    *   &.rcCst|\}}it|di}|dp|d|}dd|D|_||_t||_ t |dsrr_r_r`rrz+attempt_load_one_weight..rrrrrN)rr=rrYrrr]rrrrrrrrrrrrWrrr)rrOrrrr]ryrr_r_r`attempt_load_one_weight s   "  rcsddl}d}td}fdddD\}}}fddd D\} } } |rCd } | st?||| d%}t"urt>t?d$|d&| d%d$<t@d&d$krtt?d&|| d%d&<tJtLtMtNtOtPhvr|_TnUtUurAd$fd0d/|DnBtVurЈd}|}||gd$dn*tWur܈|d}nt tXtYhvrd}|}gd$dn|}|d$krt&j'jZfd1dt[|Dn}t6d%d2\d3d}tId4d|]D|_^|||__|_`|_a|rVt dt6|d|d|j^d5d|dt6d|Bfd6dt5|t@rf|gn|D| S|dkrwgS|qt&j'jZ| tb|fS)7a/ Parse a YOLO model.yaml dictionary into a PyTorch model. Args: d (dict): Model dictionary. ch (int): Input channels. verbose (bool): Whether to print model details. Returns: (tuple): Tuple containing the PyTorch model and sorted list of output layers. rNTinfc3s|]}|VqdSrrrdr_r`rCrzparse_model..)r activationscalesc3s|] }|dVqdS)g?Nrrrr_r`rDs)depth_multiplewidth_multipler.rz'no model scale passed. Assuming scale='z'.z activation:rrrz>3fromz>20nrz>10rrz<45 argumentsz<30rhrrgznn.rztorchvision.ops. torchvisionrurrFmlxlx)Tg333333?rc3s|]}|VqdSrr_rrr_r`rrcrfr_r_rrr_r`rprzparse_model..crfr_r_rrr_r`rprc3s|]}VqdSrr_)rlr)r]rr_r`rrrz __main__.css|]}|VqdSr)numelrr_r_r`rrrc3s |] }|dkr|VqdS)rhNr_r)r~r_r`rr)castrrrkeysr?rrr! default_actrr@ frozensetrr#r)rr(rrrrr%r'rrrrrr2r rrrrr r rrrConvTranspose2dr&rr0r r6rrrr __import__opsglobalsrWstr contextlibsuppressr4locals literal_evalrroundrLminr{insertextendrr+r*r4 BatchNorm2dr rr$r9r:r7r;r/r r,r<r}legacyr5rrr8r-r<rreplacerzrr~rzrsorted)rrrr r max_channelsractrdepthwidthr.rlayersrc2 base_modulesrepeat_modulesrzrrman_c1cmm_rr_)r]rrr~rr`r2s   2      !"&&   &$    B                  2:*   rcCst|}|jdddDvr+tdd|j}td|jd|d|||j}td d t|}t |d d p=t |}t |}t ||d <t||d<|S)z Load a YOLOv8 model from a YAML file. Args: path (str | Path): Path to the YAML file. Returns: (dict): Model dictionary. css*|]}dD] }d||dVqqdS))ryolov6Nr_)rlr\rr_r_r`rs(z"yaml_model_load..nsmlxz(\d+)([nslmx])6(.+)?$z \1\2-p6\3z8Ultralytics YOLO P6 models now use -p6 suffix. Renaming z to rz(\d+)([nslmx])(.+)?$z\1\3F)hardr yaml_file) rstemresubr?r with_namerrrErBguess_model_scale)pathnew_stem unified_pathr0rr_r_r`rs   rcCs0z tdt|jdWStyYdSw)a  Extract the size character n, s, m, l, or x of the model's scale from the model path. Args: model_path (str | Path): The path to the YOLO model's YAML file. Returns: (str): The size character of the model's scale (n, s, m, l, or x). zyolo(e-)?[v]?\d+([nslmx])rr)r2searchrr1groupAttributeError) model_pathr_r_r`r5s   r5c Csdd}t|tr"tt ||WdS1swYt|tjjrdD]}ttt|dWdS1sEwYq+dD]}tt|t|WdS1sgwYqM| D]0}t|t t fr}dSt|t rdSt|t rd St|trd St|ttttfrd Sqqt|ttfrt|}d |jvsd|jvrdSd |jvsd|jvrdSd|jvsd |jvrd Sd|jvsd |jvrd Sd |jvrd Stdd S)a Guess the task of a PyTorch model from its architecture or configuration. Args: model (torch.nn.Module | dict): PyTorch model or model configuration in YAML format. Returns: (str): Task of the model ('detect', 'segment', 'classify', 'pose', 'obb'). cSsT|ddd}|dvrdSd|vrdSd|vrdS|dkr"dS|d kr(d Sd S) zGuess from YAML dictionary.rgrhr>fcrclassify classifierr>detectsegmentposeobbN)lower)rrr_r_r`cfg2tasksz"guess_model_task..cfg2taskN)z model.argszmodel.model.argszmodel.model.model.argsr)z model.yamlzmodel.model.yamlzmodel.model.model.yamlrAr>rBrCr@z-segz-clsz-posez-obbzUnable to automatically guess model task, assuming 'task=detect'. Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'.)rWrXrr ExceptionrrModulerrr7r;rr/r r$r9r:r<rrr1partsr?r)ryrEr\rr_r_r`r sT     &  &     rr-)F)NTFr)rpickler2rCrrpathlibrrtorch.nnrultralytics.nn.autobackendrultralytics.nn.modulesrrrrr r r r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<ultralytics.utilsr=r>r?r@rArBultralytics.utils.checksrCrDrErrFrGrHrIrJrKultralytics.utils.opsrLultralytics.utils.plottingrMultralytics.utils.torch_utilsrNrOrPrQrRrSrTrUrrxrGrVrrr)r+r1rErirrrrcontextmanagerrrrrrrrrrr5rr_r_r_r`s^     :   (  PJoh`" 0  L 1 %3