o W hE@sddlZddlmZmZddlmZddlZddlmZmZm Z ddl m Z ddl m Z ddlmZmZddlmZdd lmZmZmZdd lmZd d lmZGd ddeZGdddeZGdddeZGdddeeZGdddeZdS)N)copydeepcopy)Path)YOLOConcatDatasetbuild_groundingbuild_yolo_dataset)LoadVisualPromptcheck_det_dataset)DetectionTrainerDetectionValidator) YOLOEModel) DEFAULT_CFGLOGGERRANK) de_parallel)YOLOEDetectValidatorcsPeZdZdZeddffdd ZdddZdd Zdd d Zfd dZ Z S) YOLOETrainerz"A base trainer for YOLOE training.Ncs(|duri}d|d<t|||dS)aQ Initialize the YOLOE Trainer with specified configurations. This method sets up the YOLOE trainer with the provided configuration and overrides, initializing the training environment, model, and callbacks for YOLOE object detection training. Args: cfg (dict): Configuration dictionary with default training settings from DEFAULT_CFG. overrides (dict, optional): Dictionary of parameter overrides for the default configuration. _callbacks (list, optional): List of callback functions to be applied during training. NF overlap_masksuper__init__selfcfg overrides _callbacks __class__W/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/models/yolo/yoloe/train.pyrs zYOLOETrainer.__init__TcCsNtt|tr |dn||jdt|jdd|otdkd}|r%|||S)aR Return a YOLOEModel initialized with the specified configuration and weights. Args: cfg (dict | str | None): Model configuration. Can be a dictionary containing a 'yaml_file' key, a direct path to a YAML file, or None to use default configuration. weights (str | Path | None): Path to pretrained weights file to load into the model. verbose (bool): Whether to display model information during initialization. Returns: (YOLOEModel): The initialized YOLOE model. Notes: - The number of classes (nc) is hard-coded to a maximum of 80 following the official configuration. - The nc parameter here represents the maximum number of different text samples in one image, rather than the actual number of classes. yaml_filechannelsncPchr$verbose)r isinstancedictdataminrload)rrweightsr)modelr r r! get_model(s  zYOLOETrainer.get_modelcC"d|_t|j|jt|j|jdSz7Returns a DetectionValidator for YOLO model validation.)boxclsdfl)save_dirargsr) loss_namesr test_loaderr7rr8 callbacksrr r r! get_validatorGzYOLOETrainer.get_validatortrainc CsHtt|jr t|jjndd}t|j|||j||dk||dkdS)a Build YOLO Dataset. Args: img_path (str): Path to the folder containing images. mode (str): `train` mode or `val` mode, users are able to customize different augmentations for each mode. batch (int, optional): Size of batches, this is for `rect`. Returns: (Dataset): YOLO dataset configured for training or validation. r valr?)moderectstride multi_modal)maxintr0rrDrr8r,)rimg_pathrBbatchgsr r r! build_datasetNs$ zYOLOETrainer.build_datasetcst|}|S)KProcess batch for training, moving text features to the appropriate device.)rpreprocess_batchrrIrr r!rM_s zYOLOETrainer.preprocess_batchNNTr?N) __name__ __module__ __qualname____doc__rrr1r=rKrM __classcell__r r rr!rs  rc@seZdZdZdddZdS)YOLOEPETrainerz,Fine-tune YOLOE model in linear probing way.NTcCs:tt|tr |dn||jd|jd|otdkd}|jd`|dus(Jd|r/|||t |jd }| |}| |||jd |jt|jdjd d d |jdjd d <t|jdjd d d |jdjd d <t|jdjd d d |jdjd d <|`||S) a Return YOLOEModel initialized with specified config and weights. Args: cfg (dict | str, optional): Model configuration. weights (str, optional): Path to pretrained weights. verbose (bool): Whether to display model information. Returns: (YOLOEModel): Initialized model with frozen layers except for specific projection layers. r"r#r$r&r'Nz7Pretrained weights must be provided for linear probing.namesrTr)r r*r+r,rr0savper.evallistvalues get_text_pe set_classesfusepercv3requires_grad_r?)rrr/r)r0rWtper r r!r1hs*     ...zYOLOEPETrainer.get_modelrO)rQrRrSrTr1r r r r!rVesrVcsleZdZdZeddffdd ZdddZdd Zfd d Zdd dZ ddZ ddZ fddZ Z S)YOLOETrainerFromScratchz Train YOLOE models from scratch.Ncs |duri}t|||dS)a Initialize the YOLOETrainerFromScratch class. This class extends YOLOETrainer to train YOLOE models from scratch. It inherits all functionality from the parent class while providing specialized initialization for training without pre-trained weights. Args: cfg (dict, optional): Configuration dictionary with training parameters. Defaults to DEFAULT_CFG. overrides (dict, optional): Dictionary of parameter overrides for configuration. _callbacks (list, optional): List of callback functions to be executed during training. Examples: >>> from ultralytics.models.yoloe.train import YOLOETrainerFromScratch >>> trainer = YOLOETrainerFromScratch() >>> trainer.train() Nrrrr r!rsz YOLOETrainerFromScratch.__init__r?c sttjr tjjndd|dkr#tj|j|ddSfdd|D}|t |dkr>t |S|dS) a Build YOLO Dataset for training or validation. This method constructs appropriate datasets based on the mode and input paths, handling both standard YOLO datasets and grounding datasets with different formats. Args: img_path (List[str] | str): Path to the folder containing images or list of paths. mode (str): 'train' mode or 'val' mode, allowing customized augmentations for each mode. batch (int, optional): Size of batches, used for rectangular training/validation. Returns: (YOLOConcatDataset | Dataset): The constructed dataset for training or validation. rr@r?F)rBrCrDc sNg|]#}t|trtj|j|ddn tj|d|ddqS)T)rDrErH json_file)rD)r*strrr8 training_datar).0im_pathrIrJrr r! s z9YOLOETrainerFromScratch.build_dataset..r) rFrGr0rrDrr8r,set_text_embeddingslenr)rrHrBrIdatasetsr rjr!rKs$ z%YOLOETrainerFromScratch.build_datasetcCsPt}|D] }t|ds q||jO}q|dj}|j||t|jdd|_dS)a Set text embeddings for datasets to accelerate training by caching category names. This method collects unique category names from all datasets, then generates and caches text embeddings for these categories to improve training efficiency. Args: datasets (List[Dataset]): List of datasets from which to extract category names. batch (int | None): Batch size used for processing. Notes: This method collects category names from datasets that have the 'category_names' attribute, then uses the first dataset's image path to determine where to cache the generated text embeddings. category_namesrztext_embeddings.pt) cache_pathN)sethasattrrorHgenerate_text_embeddingsrparenttext_embeddings)rrnrIrodatasetrHr r r!rls    z+YOLOETrainerFromScratch.set_text_embeddingscsft|}ttj|d}tfdd|Dj}| t |dd|j d}||d<|S)rLtextscsg|]}j|qSr )ru)rhtextr<r r!rksz.r& txt_feats) rrMr[ itertoolschaintorchstacktodevicereshapermshape)rrIrwryrr<r!rMs  z(YOLOETrainerFromScratch.preprocess_batch embeddings.ptcCsV|r t|S|jdusJ|jj||dd}tt||d}t|||S)am Generate text embeddings for a list of text samples. Args: texts (List[str]): List of text samples to encode. batch (int): Batch size for processing. cache_path (str | Path): Path to save/load cached embeddings. Returns: (dict): Dictionary mapping text samples to their embeddings. NT)without_reprtar) existsr|r.r0r]r+zipsqueezesave)rrwrIrprytxt_mapr r r!rss  z0YOLOETrainerFromScratch.generate_text_embeddingscsi}|jj}|ddsJd|ddsJddd|D}t|ddks7Jd t|dd d |dd dvrCd nd|dD]}|d durSqIt|d|d |d <qIdD]Afdd|D|<|d}|dur~qbt|tr|n|g}|D]}t|tsJdt |q||7<qb|dd d|d<|dd d|d<|dd d|d<||_|jj rt dd di|jd<d|jd<i|_ |dD]}|jj rd di|d<d|d<||j |d<q|d|dd fS)a Get train and validation paths from data dictionary. Processes the data configuration to extract paths for training and validation datasets, handling both YOLO detection datasets and grounding datasets. Returns: (str): Train dataset path. (str): Validation dataset path. Raises: AssertionError: If train or validation datasets are not found, or if validation has multiple datasets. r?Fztrain dataset not foundrAzvalidation dataset not foundcSs(i|]\}}|dd|dgDqS)cSsg|]}t|qSr r rhdr r r!rkszBYOLOETrainerFromScratch.get_dataset... yolo_data)get)rhkvr r r! s(z7YOLOETrainerFromScratch.get_dataset..rz6Only support validating on 1 dataset for now, but got .lvisrminivalNpath)r?rAcs g|] }|dkr dnqS)r?r rs val_splitr r!rks z7YOLOETrainerFromScratch.get_dataset..grounding_dataz:Grounding data should be provided in dict format, but got r$rWz)Overriding class names with single class.object)r8r,ritemsrmrfr*r[r+type single_clsrinforg)r final_data data_yamlr,rrgr rr! get_datasetsF(     z#YOLOETrainerFromScratch.get_datasetcCdS)z+Do not plot labels for YOLO-World training.Nr r<r r r!plot_training_labels5sz,YOLOETrainerFromScratch.plot_training_labelscsH|jjddd}||jj_t|trd|vrdnd|jj_tS)z Perform final evaluation on the validation dataset. Configures the validator with the appropriate dataset and split before running evaluation. Returns: (dict): Evaluation metrics. rArrrr)r8r, validatorr*rfsplitr final_eval)rrArr r!r9s   z"YOLOETrainerFromScratch.final_evalrP)r)rQrRrSrTrrrKrlrMrsrrrrUr r rr!rds   5rdcs0eZdZdZddZfddZddZZS)YOLOEPEFreeTrainerzTrain prompt-free YOLOE model.cCr2r3)r9r r:r7rr8r;r<r r r!r=Kr>z YOLOEPEFreeTrainer.get_validatorcstt||}|S)zaPreprocesses a batch of images for YOLOE training, adjusting formatting and dimensions as needed.)rrrMrNrr r!rMRsz#YOLOEPEFreeTrainer.preprocess_batchcCr)a8 Set text embeddings for datasets to accelerate training by caching category names. This method collects unique category names from all datasets, generates text embeddings for them, and caches these embeddings to improve training efficiency. The embeddings are stored in a file in the parent directory of the first dataset's image path. Args: datasets (List[Dataset]): List of datasets containing category names to process. batch (int): Batch size for processing text embeddings. Notes: The method creates a dictionary mapping text samples to their embeddings and stores it at the path specified by 'cache_path'. If the cache file already exists, it will be loaded instead of regenerating the embeddings. Nr )rrnrIr r r!rlWsz&YOLOEPEFreeTrainer.set_text_embeddings)rQrRrSrTr=rMrlrUr r rr!rHs  rcs:eZdZdZd fdd ZfddZfdd ZZS) YOLOEVPTrainerz&Train YOLOE model with visual prompts.r?NcsJt|||}t|tr|jD] }|jtq|S|jt|S)a  Build YOLO Dataset for training or validation with visual prompts. Args: img_path (List[str] | str): Path to the folder containing images or list of paths. mode (str): 'train' mode or 'val' mode, allowing customized augmentations for each mode. batch (int, optional): Size of batches, used for rectangular training/validation. Returns: (Dataset): YOLO dataset configured for training or validation, with visual prompts for training mode. )rrKr*rrn transformsappendr)rrHrBrIrvrrr r!rKns  zYOLOEVPTrainer.build_datasetcsPtt|jjtr|jjjD] }|jt qdS|jjjt dS)zPClose mosaic augmentation and add visual prompt loading to the training dataset.N) r_close_dataloader_mosaicr* train_loaderrvrrnrrr)rrrr r!rs z'YOLOEVPTrainer._close_dataloader_mosaiccs$t|}|d|j|d<|S)zcPreprocesses a batch of images for YOLOE training, moving visual prompts to the appropriate device.visuals)rrMr~rrNrr r!rMs zYOLOEVPTrainer.preprocess_batchrP)rQrRrSrTrKrrMrUr r rr!rks   r) rzrrpathlibrr|ultralytics.datarrrultralytics.data.augmentrultralytics.data.utilsr ultralytics.models.yolo.detectr r ultralytics.nn.tasksr ultralytics.utilsrrrultralytics.utils.torch_utilsrrArrrVrdrrr r r r!s$      Q.6#