o V hI@sddlZddlZddlZddlmZddlmZddlmZm Z ddl Z ddl m Z ddl mZmZmZddlmZmZmZmZmZmZddlmZerRd e d nd e d ZGd d d ZdS)N) HTTPStatus)Path)parse_qsurlparse) __version__)HELP_MSG HUB_WEB_ROOTPREFIX)IS_COLABLOGGERSETTINGSTQDMchecksemojis) HUBModelErrorzpython-z-colabz-localc @seZdZdZddZed,ddZddZd d Ze d d Z d dZ      d-ddZ e ddZ dejdedefddZddZ   d.ded ed!ed"ed#ed$df d%d&Ze d'edejd$dfd(d)Ze dejd$dfd*d+ZdS)/HUBTrainingSessiona HUB training session for Ultralytics HUB YOLO models. Handles model initialization, heartbeats, and checkpointing. This class encapsulates the functionality for interacting with Ultralytics HUB during model training, including model creation, metrics tracking, and checkpoint uploading. Attributes: model_id (str): Identifier for the YOLO model being trained. model_url (str): URL for the model in Ultralytics HUB. rate_limits (dict): Rate limits for different API calls (in seconds). timers (dict): Timers for rate limiting. metrics_queue (dict): Queue for the model's metrics. metrics_upload_failed_queue (dict): Queue for metrics that failed to upload. model (dict): Model data fetched from Ultralytics HUB. model_file (str): Path to the model file. train_args (dict): Arguments for training the model. client (HUBClient): Client for interacting with Ultralytics HUB. filename (str): Filename of the model. Examples: >>> session = HUBTrainingSession("https://hub.ultralytics.com/models/example-model") >>> session.upload_metrics() cCsddlm}dddd|_i|_i|_i|_d|_d|_d|_d|_ | |\}}|_ |p1t d}|r8d|ind}|||_z|rJ||WdS|j|_WdStyv|td rp|jjssttd YdSYdSYdSw) a Initialize the HUBTrainingSession with the provided model identifier. Args: identifier (str): Model identifier used to initialize the HUB training session. It can be a URL string or a model key with specific format. Raises: ValueError: If the provided model identifier is invalid. ConnectionError: If connecting with global API key is not supported. ModuleNotFoundError: If hub-sdk package is not installed. r) HUBClienti,)metricsckpt heartbeatNapi_key/models/zzPlease log in using 'yolo login API_KEY'. You can find your API Key at: https://hub.ultralytics.com/settings?tab=api+keys.)hub_sdkr rate_limits metrics_queuemetrics_upload_failed_queuetimersmodel model_url model_file train_args_parse_identifierfilenamer getclient load_model Exception startswithr authenticatedr warningr )self identifierrrmodel_id active_key credentialsr1K/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/hub/session.py__init__-s0    zHUBTrainingSession.__init__Nc CsXz||}|r|tds|||jjsJd|WStttfy+YdSw)a Create an authenticated HUBTrainingSession or return None. Args: identifier (str): Model identifier used to initialize the HUB training session. args (dict, optional): Arguments for creating a new model if identifier is not a HUB model URL. Returns: (HUBTrainingSession | None): An authenticated session or None if creation fails. rzHUB model not loaded correctlyN)r)r create_modelridPermissionErrorModuleNotFoundErrorAssertionError)clsr-argssessionr1r1r2create_session\s  z!HUBTrainingSession.create_sessioncCs|j||_|jjsttdtd|jj|_|jrDt d|jd|j d}t j |ttdd|jjd|_d S||j|jd t td |jdd S) a Load an existing model from Ultralytics HUB using the provided model identifier. Args: model_id (str): The identifier of the model to load. Raises: ValueError: If the specified HUB model does not exist. u*❌ The specified HUB model does not existrzLoading trained HUB model  🚀best weights_dirhub) download_dirNrView model at )r&rdata ValueErrorrrr5r is_trainedr infoget_weights_urlr check_filerr r!_set_train_argsstart_heartbeatrr )r,r.urlr1r1r2r'rs    $zHUBTrainingSession.load_modelc Cs|dd|dd|dd|ddt|d d t|d d d d|did|jdd dd iidd|jid}|jdrO|j|ddd<|j||jjs[dStd|jj|_ |j |j dt td|j ddS)a Initialize a HUB training session with the specified model arguments. Args: model_args (dict): Arguments for creating the model, including batch size, epochs, image size, etc. Returns: (None): If the model could not be created. batchepochsrimgszipatienceddevicecacheram) batchSizerN imageSizerPrRrTnamerC.pt.yaml) architectureparent)configdatasetlineagemetar_r\NrrrBr=)r%strr$replaceendswithrr4r5rr rJrr rFr )r, model_argspayloadr1r1r2r4s*      zHUBTrainingSession.create_modelcCsd\}}}t|jdvr|}n,|tdr0t|}t|jj}t|j}| ddgd}n t d|dtd |||fS) a Parse the given identifier to determine the type and extract relevant components. The method supports different identifier formats: - A HUB model URL https://hub.ultralytics.com/models/MODEL - A HUB model URL with API Key https://hub.ultralytics.com/models/MODEL?api_key=APIKEY - A local filename that ends with '.pt' or '.yaml' Args: identifier (str): The identifier string to be parsed. Returns: (tuple): A tuple containing the API key, model ID, and filename as applicable. Raises: HUBModelError: If the identifier format is not recognized. )NNN>rYrZrrNrzmodel='z invalid, correct format is z/models/MODEL_ID) rsuffixr)rrpathstemrqueryr%r)r-rr.r$ parsed_url query_paramsr1r1r2r#s    z$HUBTrainingSession._parse_identifiercCs|jr|jdd|_|jd|_n|jjd|_|jr)|jdn|j |_d|jvr8t dt j |jdd |_|jj |_d S) a. Initialize training arguments and create a model entry on the Ultralytics HUB. This method sets up training arguments based on the model's state and updates them with any additional arguments provided. It handles different states of the model, such as whether it's resumable, pretrained, or requires specific file setup. Raises: ValueError: If the model is already trained, if required dataset information is missing, or if there are issues with the provided training arguments. T)rCresumelastr"r\rCzDDataset may still be processing. Please wait a minute and try again.F)verboseN)r is_resumableget_dataset_urlr"rGr!rCr% is_pretrainedget_architecturerDrcheck_yolov5u_filenamer5r.r,r1r1r2rIs  z"HUBTrainingSession._set_train_argsrTc s<f dd} |rtj| dddS| S)av Attempt to execute `request_func` with retries, timeout handling, optional threading, and progress tracking. Args: request_func (callable): The function to execute. retry (int): Number of retry attempts. timeout (int): Maximum time to wait for the request to complete. thread (bool): Whether to run the request in a separate thread. verbose (bool): Whether to log detailed messages. progress_total (int, optional): Total size for progress tracking. stream_response (bool, optional): Whether to stream the response. *args (Any): Additional positional arguments for request_func. **kwargs (Any): Additional keyword arguments for request_func. Returns: (requests.Response | None): The response object if thread=False, otherwise None. c stt}d}tdD]}t|kr"ttdtni}|dur?ttdttd|q rH|nrO|t j |j kr\t j krjnn  drfi_|S|dkr|}rtt|dtd |j d |j sttd td |j ntd|q |dur drj d|S) zMAttempt to call `request_func` with retries, timeout, and optional threading.NzTimeout for request reached. z'Received no response from the request. rr z ()zRequest failed. )timeranger r+r rsleep_show_upload_progress_iterate_contentrOK status_codeMULTIPLE_CHOICESr%r_get_failure_message _should_retryupdate)t0responseimessage r:kwargsprogress_total request_funcretryr,stream_responsetimeoutrnr1r2 retry_requests<  " z7HUBTrainingSession.request_queue..retry_requestT)targetdaemonN) threadingThreadstart) r,rrrthreadrnrrr:rrr1rr2 request_queues-z HUBTrainingSession.request_queuecCstjtjtjh}||vS)a  Determine if a request should be retried based on the HTTP status code. Args: status_code (int): The HTTP status code from the response. Returns: (bool): True if the request should be retried, False otherwise. )rREQUEST_TIMEOUT BAD_GATEWAYGATEWAY_TIMEOUT)r retry_codesr1r1r2rIs  z HUBTrainingSession._should_retryrrrcCs||jr|rd|d|dSdS|jtjkr.|j}d|dd|dd |d dSz |d d WStyAYd Sw)aP Generate a retry message based on the response status code. Args: response (requests.Response): The HTTP response object. retry (int): The number of retry attempts allowed. timeout (int): The maximum timeout duration. Returns: (str): The retry message. z Retrying zx for zs.rSzRate limit reached (zX-RateLimit-Remaining/zX-RateLimit-Limitz). Please retry after z Retry-AfterrzNo JSON message.zUnable to read JSON.)rrrTOO_MANY_REQUESTSheadersjsonr%AttributeError)r,rrrrr1r1r2r[s   z'HUBTrainingSession._get_failure_messagecCs|j|jj|jddS)z(Upload model metrics to Ultralytics HUB.T)rr)rrupload_metricsrcopyrtr1r1r2rusz!HUBTrainingSession.upload_metricsFepochweightsis_bestmapfinalreturnc Cst|}|s3|d|j}|r&|r&ttdt||n ttd|ddS|j |j j |t ||||dd| |rJ| jnddd dS) a Upload a model checkpoint to Ultralytics HUB. Args: epoch (int): The current training epoch. weights (str): Path to the model weights file. is_best (bool): Indicates if the current model is the best one so far. map (float): Mean average precision of the model. final (bool): Indicates if the model is the final model after training. rma! Model 'best.pt' not found, copying 'last.pt' to 'best.pt' and uploading. This often happens when resuming training in transient environments like Google Colab. For more reliable training, consider using Ultralytics HUB Cloud. Learn more at https://docs.ultralytics.com/hub/cloud-training.z# Model upload issue. Missing model .N iT) rrrrrrrrrr)ris_file with_namerfr r+r shutilrrr upload_modelrastatst_size)r,rrrrrrmr1r1r2rys.  zHUBTrainingSession.upload_modelcontent_lengthcCsVt|dddd}|jddD] }|t|qWddS1s$wYdS)a! Display a progress bar to track the upload progress of a file download. Args: content_length (int): The total size of the content to be downloaded in bytes. response (requests.Response): The response object from the file download request. BT)totalunit unit_scale unit_divisor chunk_sizeN)r iter_contentrlen)rrpbarrCr1r1r2r}s  "z(HUBTrainingSession._show_upload_progresscCs|jddD]}qdS)z Process the streamed HTTP response data. Args: response (requests.Response): The response object from the file download request. rrN)r)r_r1r1r2r~sz#HUBTrainingSession._iterate_content)N)rruTTNN)FrF)__name__ __module__ __qualname____doc__r3 classmethodr<r'r4 staticmethodr#rIrrrequestsResponseintrrraboolfloatrr}r~r1r1r1r2rsR/ , # R  / r)rrrzhttprpathlibr urllib.parserrr ultralyticsrultralytics.hub.utilsrrr ultralytics.utilsr r r r rrultralytics.utils.errorsr AGENT_NAMErr1r1r1r2s