o W ht @sddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl m Z ddlZddlZddlmZddlmZddlmmZddlmZddlmZmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z%zddl&Z&Wn e'ydZ&Ynwe%ejd Z(e%ejd Z)e%ejd Z*e%ejd Z+e%e!dZ,e%e!dZ-e%e!dZ.e%e!dZ/e"re%ejdre0dede1fddZ2ddZ3dgde4de5fddZ6ddZ7d d!Z8dhd%d&Z9d'd(Z:d)d*Z;d+d,Zd2d3Z?d4d5Z@djd6d7ZAdjd8d9ZBd:d;ZCdkd>d?ZDdldAdBZEdCdDZFdmdEdFZGdGdHZHdIdJZIdndMdNZJdodOdPZKdQdRZLGdSdTdTZMdpdVe e5e fdWe5dXeNdYeNfdZd[ZOd\d]ZPedqd^d_ZQdrdadbZRGdcddddZSGdedfdfejTZUdS)sN)contextmanager)deepcopy)datetime)Path)Union) __version__)DEFAULT_CFG_DICTDEFAULT_CFG_KEYSLOGGER NUM_THREADSPYTHON_VERSIONTORCHVISION_VERSIONWINDOWScolorstr) check_versionz1.9.0z1.13.0z2.0.0z2.4.0z0.10.0z0.11.0z0.13.0z0.18.0z==2.4.0zKnown issue with torch==2.4.0 on Windows with CPU, recommend upgrading to torch>=2.4.1 to resolve https://github.com/ultralytics/ultralytics/issues/15049 local_rankccstot}|otdk}|r$|dvr$|r tj|gdntdV|r<|dkr>|r6tj|gdntdSdSdS)zjEnsures all processes in distributed training wait for the local master (rank 0) to complete a task first.nccl>r) device_idsNr)dist is_availableis_initialized get_backendbarrier)r initializeduse_idsrQ/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/utils/torch_utils.pytorch_distributed_zero_first6s  rcCs dd}|S)zXApplies torch.inference_mode() decorator if torch>=1.9.0 else torch.no_grad() decorator.cSs,trtr|Strtj|Stj|S)zNApplies appropriate torch decorator for inference mode based on torch version.) TORCH_1_9torchis_inference_mode_enabledinference_modeno_grad)fnrrrdecorateFs z&smart_inference_mode..decorater)r%rrrsmart_inference_modeCsr&cudaenableddevicecCs"tr tjj||dStjj|S)a] Get the appropriate autocast context manager based on PyTorch version and AMP setting. This function returns a context manager for automatic mixed precision (AMP) training that is compatible with both older and newer versions of PyTorch. It handles the differences in the autocast API between PyTorch versions. Args: enabled (bool): Whether to enable automatic mixed precision. device (str, optional): The device to use for autocast. Defaults to 'cuda'. Returns: (torch.amp.autocast): The appropriate autocast context manager. Notes: - For PyTorch versions 1.13 and newer, it uses `torch.amp.autocast`. - For older versions, it uses `torch.cuda.autocast`. Examples: >>> with autocast(enabled=True): ... # Your mixed precision operations here ... pass )r() TORCH_1_13r ampautocastr')r(r)rrrr,Psr,cCsddlm}d|vrLz8ddl}d}|}||d|vr!|dn |d|vr+|dn|dd}|d d d d d d |d<Wn tyKYnw|ddS) z=Return a string with system CPU information, i.e. 'Apple M2'.r)PERSISTENT_CACHEcpu_infoN) brand_raw hardware_rawarch_string_rawunknownz(R)zCPU z@ )ultralytics.utilsr-cpuinfo get_cpu_infogetreplace Exception)r-r7kinfostringrrrr8ms 8$  r8cCs&tj|}|jd|jdddS)zGReturn a string with system GPU information, i.e. 'Tesla T4, 15102MiB'., iz.0fMiB)r r'get_device_propertiesname total_memory)index propertiesrrr get_gpu_info~s rFr5FTc Cst|tjst|dst|dr|Sdtdtdtjd}t|}dD]}||d}q+|d k}|d v}|s@|rFd t j d <n_|r|d krNd}d|vr_d dd| dD}t j d d}|t j d <tjr|tjt| dkst|tjdkrdnd} td|dtjdtjd|d| |s|stjr|r| dnd} t| } | dkr|dkrtd|dkr|| dkrtd|d| d|| | d|| | | d | d! dt|d} t| D]\} }|| dkr dn| d"|d#t| d$7}qd%}n$|r5tr5tjjr5|d&td$7}d'}n |d(td$7}d }|d)vrJtt|rYt|rT|n|t|S)*a Select the appropriate PyTorch device based on the provided arguments. The function takes a string specifying the device or a torch.device object and returns a torch.device object representing the selected device. The function also validates the number of available devices and raises an exception if the requested device(s) are not available. Args: device (str | torch.device, optional): Device string or torch.device object. Options are 'None', 'cpu', or 'cuda', or '0' or '0,1,2,3'. Defaults to an empty string, which auto-selects the first available GPU, or CPU if no GPU is available. batch (int, optional): Batch size being used in your model. Defaults to 0. newline (bool, optional): If True, adds a newline at the end of the log string. Defaults to False. verbose (bool, optional): If True, logs the device information. Defaults to True. Returns: (torch.device): Selected device. Raises: ValueError: If the specified device is not available or if the batch size is not a multiple of the number of devices when using multiple GPUs. Examples: >>> select_device("cuda:0") device(type='cuda', index=0) >>> select_device("cpu") device(type='cpu') Note: Sets the 'CUDA_VISIBLE_DEVICES' environment variable for specifying which GPUs to use. tpuintelz Ultralytics u 🚀 Python-z torch- )zcuda:none()[]'rIr5cpu>mps:0mpsz-1CUDA_VISIBLE_DEVICESr'0,cSsg|]}|r|qSrr.0xrrr sz!select_device..Nrz}See https://pytorch.org/get-started/locally/ for up-to-date torch install instructions if no CUDA devices are seen by torch. zInvalid CUDA 'device=z' requested. Use 'device=cpu' or pass valid CUDA device(s) if available, i.e. 'device=0' or 'device=0,1,2,3' for Multi-GPU. torch.cuda.is_available(): z torch.cuda.device_count(): z% os.environ['CUDA_VISIBLE_DEVICES']:  r2znAutoBatch with batch<1 not supported for Multi-GPU training, please specify a valid batch size, i.e. batch=16.z'batch=z"' must be a multiple of GPU count z . Try 'batch=z ' or 'batch=z/', the nearest batch sizes evenly divisible by .zCUDA:z (z) zcuda:0zMPS (rRzCPU (>rPrR) isinstancer r)str startswithrr lowerr:osenvironjoinsplitr9r'r device_countlenr r= ValueError enumeraterF TORCH_2_0backendsrRr8set_num_threadsr rstrip)r)batchnewlineverbosesremoverPrRvisibleinstalldevicesnspaceidargrrr select_devices(!   "  .   rycCstjr tjtS)zPyTorch-accurate time.)r r'r synchronizetimerrrr time_syncs  r|c Cstj|j|j|j|j|j|j|jdd d |j j }|j |jd}t|j t|j|j}|j t|| |j j|jdurXtj|j jd|j j|j j dn|j}|j|j |jt|j|j}|jt||ddd||S) z'Fuse Conv2d() and BatchNorm2d() layers.T) kernel_sizestridepaddingdilationgroupsbiasFrNr)dtyper)r2)nnConv2d in_channels out_channelsr}r~rrrrequires_grad_toweightr)viewr diagdivsqrteps running_varcopy_mmshaperzerosrmul running_meanreshape)convbn fusedconvw_convw_bnb_convb_bnrrrfuse_conv_and_bns.    (&rc Cstj|j|j|j|j|j|j|j|j dd d |j j }|j |jd}t|j t|j|j}|j t|||j j|jdurWtj|j jd|j j dn|j}|j|j |jt|j|j}|jt||ddd||S)z0Fuse ConvTranspose2d() and BatchNorm2d() layers.T)r}r~routput_paddingrrrFrNr2r))rConvTranspose2drrr}r~rrrrrrrr)rr rrrrrrrrrrrrr)deconvr fuseddconvw_deconvrrrrrrfuse_deconv_and_bns*  *(&rcCs,|sdSt|}t|}tddd|D}t|}|rddddd d d d d ddd dd dd }t|t| D]y\} \} } | dd} | j j } t| j r| D]<\} }t| d| d| d| d |jd |dtt|jd |d|dt|j ddd q^qEt| d| d| d dd ddtgd dd dd dd qEt||}t|dddrd nd}|rd!|d"d#nd}t|d$dpt|d%id$d}t|j d&d'pd(}t|d)|d*|d+d,|d+d-|d+d.| ||||fS)/a Print and return detailed model information layer by layer. Args: model (nn.Module): Model to analyze. detailed (bool, optional): Whether to print detailed layer information. Defaults to False. verbose (bool, optional): Whether to print model information. Defaults to True. imgsz (int | List, optional): Input image size. Defaults to 640. Returns: (Tuple[int, int, int, float]): Number of layers, parameters, gradients, and GFLOPs. N collectionscss*|]\}}t|jdkr||fVqdS)rN)re_modules)rWrtmrrr C(zmodel_info..layerz>5rBz>40typez>20gradientz>10 parametersz>12rmusigmaz module_list.r5z>5gr[z>12gz>10.3gztorch.z>15Fr-is_fusedcSsdS)NFrrrrrTszmodel_info..z (fused)r?.1fz GFLOPs yaml_fileyamlyoloYOLOModelz summaryz: rUz layers, z parameters, z gradients)get_num_paramsget_num_gradients __import__ OrderedDict named_modulesrer r=rgitemsr: __class____name__ _parametersnamed_parameters requires_gradnumelr]listrmeanstdr get_flopsgetattrr9rstem)modeldetailedrnimgszn_pn_glayersn_lhrvmnrmtpnpflopsfusedfsr model_namerrr model_info2s4 4   hF  2 rcCtdd|DS)z6Return the total number of parameters in a YOLO model.cs|]}|VqdSNrrVrrrr^z!get_num_params..sumrrrrrr\rcCr)zEReturn the total number of parameters with gradients in a YOLO model.css|] }|jr|VqdSr)rrrVrrrrcsz$get_num_gradients..rrrrrrarrcCsp|jjrddlm}||jg|jdd}|dnt|jt t |jdd}t |j j dd|d<|S) a[ Return model info dict with useful model information. Args: trainer (ultralytics.engine.trainer.BaseTrainer): The trainer object containing model and validation data. Returns: (dict): Dictionary containing model parameters, GFLOPs, and inference speeds. Examples: YOLOv8n info for loggers >>> results = { ... "model/parameters": 3151904, ... "model/GFLOPs": 8.746, ... "model/speed_ONNX(ms)": 41.244, ... "model/speed_TensorRT(ms)": 3.211, ... "model/speed_PyTorch(ms)": 18.755, ...} r) ProfileModelsrz model/name)zmodel/parametersz model/GFLOPs inferencezmodel/speed_PyTorch(ms)) argsprofileultralytics.utils.benchmarksrlastr)poprrroundr validatorspeed)trainerrresultsrrrmodel_info_for_loggersfs  rcCstsdSzt|}t|}t|ts||g}zAt|dr(tt|j dnd}t j d|j d||f|j d}tjt||gdddd d }||d||d|WWStyt j d|j dg|R|j d}tjt||gdddd d YWSwtyYdSw) a  Calculate FLOPs (floating point operations) for a model in billions. Attempts two calculation methods: first with a stride-based tensor for efficiency, then falls back to full image size if needed (e.g., for RTDETR models). Returns 0.0 if thop library is unavailable or calculation fails. Args: model (nn.Module): The model to calculate FLOPs for. imgsz (int | List[int], optional): Input image size. Defaults to 640. Returns: (float): The model FLOPs in billions. r~ r2rFinputsrnreAr3)thop de_parallelnextrr\rhasattrmaxintr~r emptyrr)rrr;)rrrr~imrrrrrs&  ""   ( rc CsjtsdSt|}t|}t|ts||g}z[t|dr'tt|j dndd}t j d|j d||f|j d}t jjdd }||Wd n1sPwYtd d |Dd }||d ||d|}W|Styt j d|j dg|R|j d}t jjdd }||Wd n1swYtdd |Dd }Y|Sw)a7 Compute model FLOPs using torch profiler (alternative to thop package, but 2-10x slower). Args: model (nn.Module): The model to calculate FLOPs for. imgsz (int | List[int], optional): Input image size. Defaults to 640. Returns: (float): The model's FLOPs in billions. rr~rr3r2rT) with_flopsNcs|]}|jVqdSrrrVrrrrz0get_flops_with_torch_profiler..rrcsrrrrVrrrrr)rhrrrr\rrrrr~r rrr)profilerrr key_averagesr;)rrrr~rprofrrrrget_flops_with_torch_profilers.   &   r cCs`|D])}t|}|tjurq|tjurd|_d|_q|tjtjtj tj tj hvr-d|_ qdS)z*Initialize model weights to random values.gMbP?gQ?TN) modulesrrr BatchNorm2drmomentum Hardswish LeakyReLUReLUReLU6SiLUinplace)rrtrrrinitialize_weightss   r?rcsdkr|S|jdd\}}t|t|f}tj||ddd}|s4fdd||fD\}}tj|d ||d d ||d gd d S) a Scales and pads an image tensor, optionally maintaining aspect ratio and padding to gs multiple. Args: img (torch.Tensor): Input image tensor. ratio (float, optional): Scaling ratio. Defaults to 1.0. same_shape (bool, optional): Whether to maintain the same shape. Defaults to False. gs (int, optional): Grid size for padding. Defaults to 32. Returns: (torch.Tensor): Scaled and padded image tensor. rr3NbilinearF)sizemode align_cornersc3s&|]}t|VqdSr)mathceilrVgsratiorrrs$zscale_img..rr2gS㥛?)value)rrF interpolatepad)imgr same_shaperrwrorrr scale_imgs (r'rcCsH|jD]\}}t|r||vs|ds||vrqt|||qdS)a Copies attributes from object 'b' to object 'a', with options to include/exclude certain attributes. Args: a (object): Destination object to copy attributes to. b (object): Source object to copy attributes from. include (tuple, optional): Attributes to include. If empty, all attributes are included. Defaults to (). exclude (tuple, optional): Attributes to exclude. Defaults to (). _N)__dict__rrer^setattr)abincludeexcluder<vrrr copy_attrs  "r0cCsLtrtddttjDdStjjddd}dddd d d |d S) z Return the second-most recent ONNX opset version supported by this version of PyTorch, adjusted for maturity. Returns: (int): The ONNX opset version. css(|]}d|vrt|ddVqdS)symbolic_opsetN)r)rWr<rrrr s&z#get_latest_opset..r2r[rr2 )z1.12z1.11z1.10z1.9z1.8)r*rvarsr onnxproducer_versionrsplitr9)versionrrrget_latest_opsetsr;csfdd|DS)ab Returns a dictionary of intersecting keys with matching shapes, excluding 'exclude' keys, using da values. Args: da (dict): First dictionary. db (dict): Second dictionary. exclude (tuple, optional): Keys to exclude. Defaults to (). Returns: (dict): Dictionary of intersecting keys with matching shapes. csDi|]\}vrtfddDr|jjkr|qS)c3s|]}|vVqdSrrrVr<rrrrz-intersect_dicts...)allr)rWr/dbr.r<r sDz#intersect_dicts..)r)dar?r.rr>rintersect_dictss rBcCst|tjjtjjfS)z Returns True if model is of type DP or DDP. Args: model (nn.Module): Model to check. Returns: (bool): True if model is DataParallel or DistributedDataParallel. )r\rparallel DataParallelDistributedDataParallelrrrr is_parallel"s rFcCst|r|jS|S)z De-parallelize a model: returns single-GPU model if model is of type DP or DDP. Args: model (nn.Module): Model to de-parallelize. Returns: (nn.Module): De-parallelized model. )rFmodulerrrrr/s rrdcsfddS)a Returns a lambda function for sinusoidal ramp from y1 to y2 https://arxiv.org/pdf/1812.01187.pdf. Args: y1 (float, optional): Initial value. Defaults to 0.0. y2 (float, optional): Final value. Defaults to 1.0. steps (int, optional): Number of steps. Defaults to 100. Returns: (function): Lambda function for computing the sinusoidal ramp. cs.tdt|tjddS)Nr2r3r)rrcospirXstepsy1y2rrrHs.zone_cycle..r)rNrOrMrrLr one_cycle<s rPcCst|tj|t|tj|tj||rAtr:tjddddtj j _ dt j d<t|t j d<dStddStdS)a Initialize random number generator (RNG) seeds https://pytorch.org/docs/stable/notes/randomness.html. Args: seed (int, optional): Random seed. Defaults to 0. deterministic (bool, optional): Whether to set deterministic algorithms. Defaults to False. T) warn_onlyz:4096:8CUBLAS_WORKSPACE_CONFIGPYTHONHASHSEEDz3Upgrade to torch>=2.0.0 for deterministic training.N)randomseednpr manual_seedr'manual_seed_allrhuse_deterministic_algorithmsricudnn deterministicr`rar]r warningunset_deterministic)rUr[rrr init_seedsKs        r^cCs4tddtjj_tjddtjdddS)zAUnsets all the configurations applied for deterministic training.FrRNrS)r rYrirZr[r`rarrrrrr]es  r]c@s,eZdZdZdddZddZdd d Zd S)ModelEMAa Updated Exponential Moving Average (EMA) implementation. Keeps a moving average of everything in the model state_dict (parameters and buffers). For EMA details see References. To disable EMA set the `enabled` attribute to `False`. Attributes: ema (nn.Module): Copy of the model in evaluation mode. updates (int): Number of EMA updates. decay (function): Decay function that determines the EMA weight. enabled (bool): Whether EMA is enabled. References: - https://github.com/rwightman/pytorch-image-models - https://www.tensorflow.org/api_docs/python/tf/train/ExponentialMovingAverage H.?rcsLtt||_||_fdd|_|jD]}|dqd|_dS)au Initialize EMA for 'model' with given arguments. Args: model (nn.Module): Model to create EMA for. decay (float, optional): Maximum EMA decay rate. Defaults to 0.9999. tau (int, optional): EMA decay time constant. Defaults to 2000. updates (int, optional): Initial number of updates. Defaults to 0. csdt| S)Nr2)rexprKdecaytaurrrsz#ModelEMA.__init__..FTN) rrevalemaupdatesrdrrr()selfrrdrerhrrrcr__init__s    zModelEMA.__init__cCst|jr6|jd7_||j}t|}|jD]\}}|jjr5||9}|d||| 7}qdSdS)zp Update EMA parameters. Args: model (nn.Module): Model to update EMA from. r2N) r(rhrdr state_dictrgrris_floating_pointdetach)rirrwmsdr<r/rrrupdates  zModelEMA.updater process_groupreducercCs|jr t|j|||dSdS)aZ Updates attributes and saves stripped model with optimizer removed. Args: model (nn.Module): Model to update attributes from. include (tuple, optional): Attributes to include. Defaults to (). exclude (tuple, optional): Attributes to exclude. Defaults to ("process_group", "reducer"). N)r(r0rg)rirr-r.rrr update_attrs zModelEMA.update_attrN)r`rar)rrp)r __module__ __qualname____doc__rjrorsrrrrr_ms  r_best.ptfrorhreturnc Csztj|tdd}t|tsJdd|vsJdWnty<}ztd|d|iWYd}~Sd}~wwt t d d d }| d rS|d |d<t |dd rdt|dj|d_t |ddrpd|d_|d|dD]}d|_q|it| di}dD]}d||<qd|d<dd|D|d<i|||pi} t| |p|tj|p|d} td|d|rd|dndd| dd| S)a Strip optimizer from 'f' to finalize training, optionally save as 's'. Args: f (str | Path): File path to model to strip the optimizer from. Defaults to 'best.pt'. s (str, optional): File path to save the model with stripped optimizer to. If not provided, 'f' will be overwritten. updates (dict, optional): A dictionary of updates to overlay onto the checkpoint before saving. Returns: (dict): The combined checkpoint dictionary. Examples: >>> from pathlib import Path >>> from ultralytics.utils.torch_utils import strip_optimizer >>> for f in Path("path/to/model/checkpoints").rglob("*.pt"): >>> strip_optimizer(f) rP) map_locationz%checkpoint is not a Python dictionaryrz'model' missing from checkpointz Skipping z!, not a valid Ultralytics model: Nz2AGPL-3.0 License (https://ultralytics.com/license)zhttps://docs.ultralytics.com)dater:licensedocsrgr criterionF train_args) optimizer best_fitnessrgrhrepochcSsi|] \}}|tvr||qSr)r )rWr<r/rrrr@sz#strip_optimizer..g.AzOptimizer stripped from rUz saved as r5rIrMB)r loadr)r\dictr;r r\rnow isoformatrr9rrr~halfrrrrsaver`pathgetsizer=) rxrorhrXemetadatarrr<combinedmbrrrstrip_optimizersB      0rcCsT|dD]!}|D]\}}|dkr&t|tjr&|jtjur&|||<q q|S)a  Converts the state_dict of a given optimizer to FP16, focusing on the 'state' key for tensor conversions. Args: state_dict (dict): Optimizer state dictionary. Returns: (dict): Converted optimizer state dictionary with FP16 tensors. statestep)valuesrr\r Tensorrfloat32r)rkrr<r/rrr$convert_optimizer_state_dict_to_fp16s   rc csZtdd}tjr(tjz|VWtj||d<dStj||d<w|VdS)a` Monitor and manage CUDA memory usage. This function checks if CUDA is available and, if so, empties the CUDA cache to free up unused memory. It then yields a dictionary containing memory usage information, which can be updated by the caller. Finally, it updates the dictionary with the amount of memory reserved by CUDA on the specified device. Args: device (torch.device, optional): The CUDA device to query memory usage for. Defaults to None. Yields: (dict): A dictionary with a key 'memory' initialized to 0, which will be updated with the reserved memory. r)memoryrN)rr r'r empty_cachememory_reserved)r) cuda_inforrrcuda_memory_usages   & r c sg}t|tjs t|}tdddddddddddd d d ttj t|t r6|n|gD] |d _ t|t rK|n|gD]}t |d r[| |n|}t |d rrttjrrjtjurr|n|}ddgd}}} ztrtjt|gdddddnd} Wn tyd} Ynwz+zd} t|D]} t|A} t| d<|}t| d<zt|t rtdd|Dn|t| d<Wntytd| d<YnwWdn1swY| | dd7} || d| dd|7}|| d| dd|7}|rZt|$} tjjd|ttfdd|j D|tj!dWdn 1sMwY| | dd7} qdd|fD\}}t|t"j#rxtdd|$Dnd}t|d| d | d!|d"|d"t%|d t%|d |&|| | ||||gWnty}zt||&dWYd}~nd}~wwWttj qNttj wq9|S)#a; Ultralytics speed, memory and FLOPs profiler. Args: input (torch.Tensor | List[torch.Tensor]): Input tensor(s) to profile. ops (nn.Module | List[nn.Module]): Model or list of operations to profile. n (int, optional): Number of iterations to average. Defaults to 10. device (str | torch.device, optional): Device to profile on. Defaults to None. max_num_obj (int, optional): Maximum number of objects for simulation. Defaults to 0. Returns: (list): Profile results for each operation. Examples: >>> from ultralytics.utils.torch_utils import profile >>> input = torch.randn(16, 3, 640, 640) >>> m1 = lambda x: x * torch.sigmoid(x) >>> m2 = nn.SiLU() >>> profile(input, [m1, m2], n=100) # profile over 100 iterations Paramsz>12sGFLOPsz GPU_mem (GB)z>14sz forward (ms)z backward (ms)inputz>24soutputTrrr)rrrFrrr3r2csrr)r)rWyirrrrKrzprofile..nanNric3s,|]}jd|jd|VqdS)rN)r)rWrorKrrrXs*)r)rcss*|]}t|tjrt|jndVqdS)rN)r\r rtuplerrVrrrr]rcsrrrrVrrrr^r12z12.4gz>14.3fz14.4g)'r\r r)ryr r=gccollectr'rrrrrrrfloat16rrrrr;rangerr|rbackwardfloatrandnrrr~tolistrrModulerr]append)ropsrtr) max_num_objrrtftbrrmemr(rys_ins_outrrrrKrrs   ..     (   (<   -rc@s"eZdZdZdddZddZdS) EarlyStoppinga Early stopping class that stops training when a specified number of epochs have passed without improvement. Attributes: best_fitness (float): Best fitness value observed. best_epoch (int): Epoch where best fitness was observed. patience (int): Number of epochs to wait after fitness stops improving before stopping. possible_stop (bool): Flag indicating if stopping may occur next epoch. 2cCs$d|_d|_|p td|_d|_dS)z Initialize early stopping object. Args: patience (int, optional): Number of epochs to wait after fitness stops improving before stopping. rrinfFN)r best_epochrpatience possible_stop)rirrrrrjus zEarlyStopping.__init__c Cs|durdS||jks|jdkr||_||_||j}||jdk|_||jk}|rBtd}t|d|jd|jd|jd |S) a Check whether to stop training. Args: epoch (int): Current epoch of training fitness (float): Fitness value of current epoch Returns: (bool): True if training should stop, False otherwise NFrr2zEarlyStopping: z:Training stopped early as no improvement observed in last z( epochs. Best results observed at epoch z@, best model saved as best.pt. To update EarlyStopping(patience=z^) pass a new patience value, i.e. `patience=300` or use `patience=0` to disable EarlyStopping.)rrrrrr r=)rirfitnessdeltastopprefixrrr__call__s$   zEarlyStopping.__call__N)r)rrtrurvrjrrrrrrjs rcs(eZdZdZfddZddZZS)FXModela} A custom model class for torch.fx compatibility. This class extends `torch.nn.Module` and is designed to ensure compatibility with torch.fx for tracing and graph manipulation. It copies attributes from an existing model and explicitly sets the model attribute to ensure proper copying. Attributes: model (nn.Module): The original model's layers. cs tt|||j|_dS)z Initialize the FXModel. Args: model (nn.Module): The original model to wrap for torch.fx compatibility. N)superrjr0r)rirrrrrjs   zFXModel.__init__cs\g|jD]&}|jdkr"t|jtr|jn fdd|jD|qS)aa Forward pass through the model. This method performs the forward pass through the model, handling the dependencies between layers and saving intermediate outputs. Args: x (torch.Tensor): The input tensor to the model. Returns: (torch.Tensor): The output tensor from the model. rcs g|] }|dkr n|qS)rr)rWjrXrrrrYs z#FXModel.forward..)rrxr\rr)rirXrrrrforwards  , zFXModel.forward)rrtrurvrjr __classcell__rrrrrs  r)r')r5rFT)FTr)r)rFr)rr)r)rrrH)rF)rwr5Nr)rNr)Vrrr`rTr{ contextlibrcopyrrpathlibrtypingrnumpyrVr torch.distributed distributedrtorch.nnrtorch.nn.functional functionalr! ultralyticsrr6rr r r r r rrultralytics.utils.checksrr ImportErrorrr*rh TORCH_2_4TORCHVISION_0_10TORCHVISION_0_11TORCHVISION_0_13TORCHVISION_0_18r\rrr&boolr]r,r8rFryr|rrrrrrrr rr'r0r;rBrFrrPr^r]r_rrrrrrrrrrrrs        (              f" * " %"   $D<  Q6