o V h@svddlZddlmZddlmZddlmZddlmZddl Z ddl Z ddl Z ddl mZddlmZddlmZmZmZmZmZdd lmZdd lmZmZdd lmZd d lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%d dl&m'Z'd dl(m)Z)d dl*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2dZ3Gddde'Z4Gddde4Z5Gddde4Z6GdddeZ7Gddde'Z8GdddZ9dS)N) defaultdict)repeat) ThreadPool)Path)Image) ConcatDataset) LOCAL_RANKLOGGER NUM_THREADSTQDMcolorstr) Instances)resample_segmentssegments2boxes)TORCHVISION_0_18)ComposeFormat LetterBoxRandomLoadTextclassify_augmentationsclassify_transforms v8_transforms) BaseDataset)merge_multi_segment)HELP_URLcheck_file_speedsget_hashimg2label_pathsload_dataset_cache_filesave_dataset_cache_file verify_imageverify_image_labelz1.0.3csfeZdZdZdddfdd Zedfdd Zd d Zdd d ZddZ ddZ e ddZ Z S) YOLODataseta Dataset class for loading object detection and/or segmentation labels in YOLO format. This class supports loading data for object detection, segmentation, pose estimation, and oriented bounding box (OBB) tasks using the YOLO format. Attributes: use_segments (bool): Indicates if segmentation masks should be used. use_keypoints (bool): Indicates if keypoints should be used for pose estimation. use_obb (bool): Indicates if oriented bounding boxes should be used. data (dict): Dataset configuration dictionary. Methods: cache_labels: Cache dataset labels, check images and read shapes. get_labels: Returns dictionary of labels for YOLO training. build_transforms: Builds and appends transforms to the list. close_mosaic: Sets mosaic, copy_paste and mixup options to 0.0 and builds transformations. update_labels_info: Updates label format for different tasks. collate_fn: Collates data samples into batches. Examples: >>> dataset = YOLODataset(img_path="path/to/images", data={"names": {0: "person"}}, task="detect") >>> dataset.get_labels() NdetectdatataskcsN|dk|_|dk|_|dk|_||_|jr|jrJdtj|i|dS)ap Initialize the YOLODataset. Args: data (dict, optional): Dataset configuration dictionary. task (str): Task type, one of 'detect', 'segment', 'pose', or 'obb'. *args (Any): Additional positional arguments for the parent class. **kwargs (Any): Additional keyword arguments for the parent class. segmentposeobbz(Can not use both segments and keypoints.N) use_segments use_keypointsuse_obbr&super__init__selfr&r'argskwargs __class__L/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/data/dataset.pyr/Hs  zYOLODataset.__init__./labels.cachec Cs0dgi}ddddgf\}}}}}|jd|j|jd}t|j} |jdd\} } |jr:| dks6| dvr:tdt t } | j t t |j|jt|jt|jtt|jd t| t| t|jd } t| || d }|D]V\ }}}}}}}}}}||7}||7}||7}||7}|r|d|||d d dd f|d d d d f||ddd|r|||d|d||d|d|_qn|Wd n1swY|rtd||dkrt|jd|dtt|j|j|d<||||t|jf|d<||d<t|j||t|S)z Cache dataset labels, check images and read shapes. Args: path (Path): Path where to save the cache file. Returns: (dict): Dictionary containing cached labels and related information. labelsr Scanning ... kpt_shape)rr>z'kpt_shape' in data.yaml missing or incorrect. Should be a list with [number of keypoints, number of dims (2 for x,y or 3 for x,y,visible)], i.e. 'kpt_shape: [17, 3]'namesfunciterabledesctotalNrTxywh)im_fileshapeclsbboxessegments keypoints normalized bbox_format images,  backgrounds,  corrupt No labels found in z. hashresultsmsgs)prefixparentstemlenim_filesr&getr, ValueErrorrr imapr"zip label_filesr single_clsr appendrDcloser infojoinwarningrrr DATASET_CACHE_VERSION)r1pathxnmnfnencrWrDrEnkptndimpoolrVpbarrGlbrHrKkeypointnm_fnf_fne_fnc_fmsgr6r6r7 cache_labelsYsn     $ &zYOLODataset.cache_labelsc st|j|_t|jdjd}zt|d}dtks!Jdt|j|jks/JWnt t t fyD| |d}Ynw d\}}}}}|rtdvrd |d |d ||d |d }td|j|||ddrtddfdddDd} | std|dtdd| D|_dd| D} ddt| D\} } } | r| | krtd| d| d| D]}g|d<q| dkrtd |dt| S)!a9 Returns dictionary of labels for YOLO training. This method loads labels from disk or cache, verifies their integrity, and prepares them for training. Returns: (List[dict]): List of label dictionaries, each containing information about an image and its annotations. r.cacheTversionrUFrVrr:z... rPrQrRNrDrEinitialrWrScg|]}|qSr6pop.0kcacher6r7 z*YOLODataset.get_labels..)rUr|rWr9zNo images found in z#, training may not work correctly. cSg|]}|dqSrGr6rrsr6r6r7rcss2|]}t|dt|dt|dfVqdS)rIrJrKNr[rr6r6r7 s0z)YOLODataset.get_labels..cs|]}t|VqdSN)sumrrjr6r6r7rz@Box and segment counts should be equal, but got len(segments) = z, len(boxes) = z. To resolve this only boxes will be used and all segments will be removed. To avoid this please supply either a detect or segment dataset, not a detect-segment mixed dataset.rKrT)rr\rarrY with_suffixrrhrFileNotFoundErrorAssertionErrorAttributeErrorrzrrr rXr rerfrgrr`)r1 cache_pathexistsrlrkrmrnndr9lengthslen_cls len_boxes len_segmentsrsr6rr7 get_labelssB   "  zYOLODataset.get_labelsc Cs|jr#|jr |js |jnd|_|jr|js|jnd|_t||j|}n tt|j|jfddg}|t dd|j |j |j d|j |j|jrF|jndd |S)z Builds and appends transforms to the list. Args: hyp (dict, optional): Hyperparameters for transforms. Returns: (Compose): Composed transforms. F) new_shapescaleuprFT) rN normalize return_maskreturn_keypoint return_obb batch_idx mask_ratio mask_overlapbgr)augmentrectmosaicmixuprimgszrrrcrr+r,r-r overlap_maskr)r1hyp transformsr6r6r7build_transformss&  zYOLODataset.build_transformscCs"d|_d|_d|_|||_dS) Sets mosaic, copy_paste and mixup options to 0.0 and builds transformations. Args: hyp (dict): Hyperparameters for transforms. rN)r copy_pasterrr)r1rr6r6r7 close_mosaicszYOLODataset.close_mosaicc Cs|d}|dg}|dd}|d}|d}|jr dnd}t|d krGtd d |D}||kr9|d n|}tjt||d d d}n tjd |dftjd}t |||||d|d<|S)a Custom your label format here. Args: label (dict): Label dictionary containing bboxes, segments, keypoints, etc. Returns: (dict): Updated label dictionary with instances. Note: cls is not with bboxes now, classification and semantic segmentation need an independent cls label Can also support classification and semantic segmentation by adding or removing dict keys there. rJrKrLNrNrMdircsrrr)rsr6r6r7rrz1YOLODataset.update_labels_info..r)raxisr=dtype)rNrM instances) rr-r[maxnpstackrzerosfloat32r ) r1labelrJrKrLrNrMsegment_resamplesmax_lenr6r6r7update_labels_infos      zYOLODataset.update_labels_infocCsi}dd|D}|d}ttdd|D}t|D]/\}}||}|dvr1t|d}n|dkr?tjjjj |dd}|d vrIt |d}|||<qt|d |d <t t |d D] }|d ||7<q^t |d d|d <|S) Collates data samples into batches. Args: batch (List[dict]): List of dictionaries containing sample data. Returns: (dict): Collated batch with stacked tensors. cSsg|] }tt|qSr6)dictsorteditemsrbr6r6r7r%z*YOLODataset.collate_fn..rcSsg|]}t|qSr6)listvaluesrr6r6r7r'>img text_featsvisualsT) batch_first>rIr*masksrJrKrLr) keysrr` enumeratetorchrnnutilsrnn pad_sequencecatranger[)batch new_batchrrirvaluer6r6r7 collate_fns$    zYOLODataset.collate_fnr)__name__ __module__ __qualname____doc__r/rrzrrrr staticmethodr __classcell__r6r6r4r7r#.sD 0 !r#csheZdZdZdddfdd ZfddZdfd d Zed d Zed dZ e dddZ Z S)YOLOMultiModalDataseta Dataset class for loading object detection and/or segmentation labels in YOLO format with multi-modal support. This class extends YOLODataset to add text information for multi-modal model training, enabling models to process both image and text data. Methods: update_labels_info: Adds text information for multi-modal model training. build_transforms: Enhances data transformations with text augmentation. Examples: >>> dataset = YOLOMultiModalDataset(img_path="path/to/images", data={"names": {0: "person"}}, task="detect") >>> batch = next(iter(dataset)) >>> print(batch.keys()) # Should include 'texts' Nr$r%cstj|||d|dS)ax Initialize a YOLOMultiModalDataset. Args: data (dict, optional): Dataset configuration dictionary. task (str): Task type, one of 'detect', 'segment', 'pose', or 'obb'. *args (Any): Additional positional arguments for the parent class. **kwargs (Any): Additional keyword arguments for the parent class. r%Nr.r/r0r4r6r7r/Is zYOLOMultiModalDataset.__init__cs,t|}dd|jdD|d<|S)a Add texts information for multi-modal model training. Args: label (dict): Label dictionary containing bboxes, segments, keypoints, etc. Returns: (dict): Updated label dictionary with instances and texts. cSsg|] \}}|dqS/split)r_vr6r6r7rbz.r?texts)r.rr&r)r1rr9r4r6r7rUs z(YOLOMultiModalDataset.update_labels_infocsDt|}|jr tt|jddd||jd}|d||S)a" Enhances data transformations with optional text augmentation for multi-modal training. Args: hyp (dict, optional): Hyperparameters for transforms. Returns: (Compose): Composed transforms including text augmentation if applicable. rnPT max_samplespadding padding_valuer~) r.rrrminr&_get_neg_texts category_freqinsertr1rr transformr4r6r7rfs   z&YOLOMultiModalDataset.build_transformscCs|jd}dd|DS)zw Return category names for the dataset. Returns: (Set[str]): List of class names. r?cSs$h|]}|dD]}|q qSr)rstrip)rnamerr6r6r7 s$z7YOLOMultiModalDataset.category_names..)r&r)r1r?r6r6r7category_names~sz$YOLOMultiModalDataset.category_namescCsrdd|jdD}tt}|jD]#}|ddD]}|t|}|D]}|}||d7<q&qq|S)1Return frequency of each category in the dataset.cSsg|]}|dqSrr)rrr6r6r7rrz7YOLOMultiModalDataset.category_freq..r?rIr~r)r&rrintr9squeezer)r1rrrctexttr6r6r7rs  z#YOLOMultiModalDataset.category_freqrcfdd|DS)7Get negative text samples based on frequency threshold.cg|] \}}|kr|qSr6r6rrr thresholdr6r7rrz8YOLOMultiModalDataset._get_neg_texts..rrrr6rr7rz$YOLOMultiModalDataset._get_neg_textsrr) rrrrr/rrpropertyrrrrrr6r6r4r7r8s    rcseZdZdZdddfdd ZddZd d Zed fd d ZddZ dfdd Z e ddZ e ddZ edddZZS)GroundingDataseta4 Handles object detection tasks by loading annotations from a specified JSON file, supporting YOLO format. This dataset is designed for grounding tasks where annotations are provided in a JSON file rather than the standard YOLO format text files. Attributes: json_file (str): Path to the JSON file containing annotations. Methods: get_img_files: Returns empty list as image files are read in get_labels. get_labels: Loads annotations from a JSON file and prepares them for training. build_transforms: Configures augmentations for training with optional text loading. Examples: >>> dataset = GroundingDataset(img_path="path/to/images", json_file="annotations.json", task="detect") >>> len(dataset) # Number of valid images with annotations r$)r' json_filecs2|dvsJd||_tj||id|dS)a Initialize a GroundingDataset for object detection. Args: json_file (str): Path to the JSON file containing annotations. task (str): Must be 'detect' or 'segment' for GroundingDataset. *args (Any): Additional positional arguments for the parent class. **kwargs (Any): Additional keyword arguments for the parent class. >r$r(zEGroundingDataset currently only supports `detect` and `segment` tasks)r'r&N)rr.r/)r1r'rr2r3r4r6r7r/s zGroundingDataset.__init__cCsgS)a  The image files would be read in `get_labels` function, return empty list here. Args: img_path (str): Path to the directory containing images. Returns: (list): Empty list as image files are read in get_labels. r6)r1img_pathr6r6r7 get_img_filess zGroundingDataset.get_img_filescCs~tdd|D}d|jvr|dksJd Sd|jvr#|dks!Jd Sd|jvr0|dks.Jd Sd |jvr=|d ks;Jd SJ) zFVerify the number of instances in the dataset matches expected counts.css|] }|djdVqdS)rJrN)rHrrr6r6r7rsz1GroundingDataset.verify_labels..final_mixed_train_no_coco_segmi7final_mixed_train_no_cocoi+8"final_flickr_separateGT_train_segmi final_flickr_separateGT_traini FN)rr)r1r9instance_countr6r6r7 verify_labelss    zGroundingDataset.verify_labelsr8c sdgi}tdt|j }t|}Wdn1swYdd|dD}tt}|dD] }||d|q4t | d |jd D]\}} ||d } | d | d | d} } }t |j |} | sqqL|jt| g}g}i}g}| D]}|drqtj|dtjd}|dd|ddd7<|ddgt| <|ddgt| <|ddks|ddkrq| ddfdd|dD}|sq||vrt|||<||g||}|g|}||vrr|||ddurrt|ddkr||qt|ddkrDt|d}tj|ddtj| | gtjdd}n$dd|dD}tj|tjdddtj| | gtjdd}|g|}||qt|rtj|tjdntjdtjd}|rtjd d|Dtjd}d!d|D}t|ddt|fd}tj|tjd}|d| | | f|ddddf|ddddf|d"d#|d$qLt|j|d%<t |j!||t"|S)&a Loads annotations from a JSON file, filters, and normalizes bounding boxes for each image. Args: path (Path): Path where to save the cache file. Returns: (dict): Dictionary containing cached labels and related information. r9zLoading annotation file...NcSsi|] }|dd|qS)idrr6rr6r6r7 rz1GroundingDataset.cache_labels..images annotationsimage_idzReading annotations )rDrheightwidth file_nameiscrowdbboxrr=rrr>captionrOcs g|] }|d|dqS)rrr6)rr r-r6r7rs z1GroundingDataset.cache_labels..tokens_positive segmentationrr~cSsg|] }|D]}|qqSr6r6)rrjr6r6r7rr)rcSrrr6rr6r6r7rrcSs,g|]}tj|ddtjdddqS)rNrr~r=)rarrayrreshaperr6r6r7r,TrF)rGrHrIrJrKrMrNrrU)#r reopenrjsonloadrrrcr rrrrr\strrr4rfloatrflowerrr[tolistr]r concatenater5rrrr rXrh)r1rirjfr&r% img_to_annsannimg_idannsrhwrGrJrKcat2idrboxcat_namerIrrsclassesr6r.r7rzs    "  $      .*  * zGroundingDataset.cache_labelsc st|jd}zt|d}dtksJdt|jks#JWntttfy8| |d}YnwfdddDd }d d|D|_ t d vr^t d |jd ||S)z Load labels from cache or generate them from JSON file. Returns: (List[dict]): List of label dictionaries, each containing information about an image and its annotations. r{Tr|rUFcrr6rrrr6r7rArz/GroundingDataset.get_labels..)rUr|r9cSsg|]}t|dqSr)r:rr6r6r7rDrr}zLoad z from cache file )rrrrrhrrrrrzr\rr re)r1rrr9r6rr7r3szGroundingDataset.get_labelsNcs8t|}|jrtdd||jd}|d||S)a  Configures augmentations for training with optional text loading. Args: hyp (dict, optional): Hyperparameters for transforms. Returns: (Compose): Composed transforms including text augmentation if applicable. rTrr~)r.rrrrrrrr4r6r7rIs   z!GroundingDataset.build_transformscCsdd|jDS)z.Return unique category names from the dataset.cSs,h|]}|dD] }|D]}|q qqS)r)r)rrr r r6r6r7rdr6z2GroundingDataset.category_names..)r9r1r6r6r7raszGroundingDataset.category_namescCsHtt}|jD]}|dD]}|D]}|}||d7<qq q|S)rrr)rrr9r)r1rrr r r6r6r7rfs  zGroundingDataset.category_freqrcr )r crr6r6rrr6r7rtrz3GroundingDataset._get_neg_texts..rrr6rr7rqrzGroundingDataset._get_neg_textsrr)rrrrr/rr"rrzrrrrrrrrr6r6r4r7rs [   rc@s$eZdZdZeddZddZdS)YOLOConcatDataseta Dataset as a concatenation of multiple datasets. This class is useful to assemble different existing datasets for YOLO training, ensuring they use the same collation function. Methods: collate_fn: Static method that collates data samples into batches using YOLODataset's collation function. Examples: >>> dataset1 = YOLODataset(...) >>> dataset2 = YOLODataset(...) >>> combined_dataset = YOLOConcatDataset([dataset1, dataset2]) cCs t|S)r)r#r)rr6r6r7rs zYOLOConcatDataset.collate_fncCs&|jD] }t|ds q||qdS)rrN)datasetshasattrr)r1rdatasetr6r6r7rs   zYOLOConcatDataset.close_mosaicN)rrrrrrrr6r6r6r7rKws   rKcs eZdZdZfddZZS)SemanticDatasetzSemantic Segmentation Dataset.cstdS)z$Initialize a SemanticDataset object.NrrJr4r6r7r/szSemanticDataset.__init__)rrrrr/rr6r6r4r7rOsrOc@s8eZdZdZdddZddZdefd d Zd d Zd S)ClassificationDataseta~ Extends torchvision ImageFolder to support YOLO classification tasks. This class offers functionalities like image augmentation, caching, and verification. It's designed to efficiently handle large datasets for training deep learning models, with optional image transformations and caching mechanisms to speed up training. Attributes: cache_ram (bool): Indicates if caching in RAM is enabled. cache_disk (bool): Indicates if caching on disk is enabled. samples (list): A list of tuples, each containing the path to an image, its class index, path to its .npy cache file (if caching on disk), and optionally the loaded image array (if caching in RAM). torch_transforms (callable): PyTorch transforms to be applied to the images. root (str): Root directory of the dataset. prefix (str): Prefix for logging and cache filenames. Methods: __getitem__: Returns subset of data and targets corresponding to given indices. __len__: Returns the total number of samples in the dataset. verify_images: Verifies all images in dataset. Frc CsBddl}tr|jj|dd|_n|jj|d|_|jj|_|jj|_|r9|jdkr9|jdtt |j|j|_|rBt |dnd|_ |j dupRt |j d k|_|jr_td d |_t |j d k|_||_d d|jD|_d|jdf}|rt|j||j|j|j|j|j|j|jd |_dSt|jd|_dS)aI Initialize YOLO object with root, image size, augmentations, and cache settings. Args: root (str): Path to the dataset directory where images are stored in a class-specific folder structure. args (Namespace): Configuration containing dataset-related settings such as image size, augmentation parameters, and cache settings. augment (bool, optional): Whether to apply augmentations to the dataset. prefix (str, optional): Prefix for logging and cache filenames, aiding in dataset identification. rNT)root allow_empty)rQg?z: rramzClassification `cache_ram` training has known memory leak in https://github.com/ultralytics/ultralytics/issues/9824, setting `cache_ram=False`.FdiskcSs*g|]}t|t|dddgqS)rz.npyN)rrrrr6r6r7rs*z2ClassificationDataset.__init__..) sizescalehflipvfliperasing auto_augmenthsv_hhsv_shsv_v)rU) torchvisionrrL ImageFolderbasesamplesrQfractionroundr[r rXrr:r< cache_ramr rg cache_disk verify_imagesrVrrfliplrflipudrYrZr[r\r]rtorch_transforms)r1rQr2rrXr^rVr6r6r7r/sD       zClassificationDataset.__init__cCs|j|\}}}}|jr|durt|}|j|d<n|jr7|s1tj|t|ddt |}nt|}t t |tj }||}||dS)z Returns subset of data and targets corresponding to given indices. Args: i (int): Index of the sample to retrieve. Returns: (dict): Dictionary containing the image and its class index. Nr>F) allow_pickle)rrI)rardcv2imreadrerrsaveas_posixr9r fromarraycvtColor COLOR_BGR2RGBri)r1rr?r1fnimsampler6r6r7 __getitem__s     z!ClassificationDataset.__getitem__returncCs t|jS)z2Return the total number of samples in the dataset.)r[rarJr6r6r7__len__s zClassificationDataset.__len__c Cs&|jd|jd}t|jd}z^tdd|jddD|jdt|}|d tks0J|d td d|jDks@J| d \}}}}t d vrn|d|d|d}t d|||d|drnt d|d|WStttfyddggif\}}} }} ttO} | jtt|jt|jd} t | |t|jd} | D](\}}}}|r|||r| |||7}||7}|d|d|d| _q| Wdn1swY| rt d| tdd|jD| d <||t||f| d <| | d<t|j|| t|YSw)z Verify all images in dataset. Returns: (list): List of valid samples after verification. r:r;r{cSsg|]\}}|qSr6r6)rfilerr6r6r7rrz7ClassificationDataset.verify_images..Nr2)rXr|rUcSrr3r6rr6r6r7r rrVr}rOrPrRrrWrSrr@rCcSrr3r6rr6r6r7r:r)rXrQrrrrarrhrrrr r rerfrrrrr r_r!r`rr[rcrDrdr )r1rDrirrlrnrrarrWrjrqrVrrrtrvrxryr6r6r7rfsJ"      z#ClassificationDataset.verify_imagesN)Fr) rrrrr/rurrwrfr6r6r6r7rPs  4 rP):r8 collectionsr itertoolsrmultiprocessing.poolrpathlibrrknumpyrrPILrtorch.utils.datarultralytics.utilsrr r r r ultralytics.utils.instancer ultralytics.utils.opsrrultralytics.utils.torch_utilsrrrrrrrrrr`r converterrrrrrrrr r!r"rhr#rrrKrOrPr6r6r6r7s8        $ (  d\+