o W h @sddlZddlZddlZddlZddlmZddlmmZ ddl m Z dZ ddZ dddZd d Zdd d Zdejdejdejdejdejf ddZdS)N)uniform_)#multi_scale_deformable_attn_pytorchinverse_sigmoidcstfddt|DS)a Create a list of cloned modules from the given module. Args: module (nn.Module): The module to be cloned. n (int): Number of clones to create. Returns: (nn.ModuleList): A ModuleList containing n clones of the input module. Examples: >>> import torch.nn as nn >>> layer = nn.Linear(10, 10) >>> clones = _get_clones(layer, 3) >>> len(clones) 3 csg|]}tqS)copydeepcopy).0_modulerP/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/nn/modules/utils.py !sz_get_clones..)nn ModuleListrange)r nrr r _get_clonessr{Gz?cCsttd|| S)a Initialize conv/fc bias value according to a given probability value. This function calculates the bias initialization value based on a prior probability using the inverse error function. It's commonly used in object detection models to initialize classification layers with a specific positive prediction probability. Args: prior_prob (float, optional): Prior probability for bias initialization. Returns: (float): Bias initialization value calculated from the prior probability. Examples: >>> bias = bias_init_with_prob(0.01) >>> print(f"Bias initialization value: {bias:.4f}") Bias initialization value: -4.5951 )floatnplog) prior_probrrr bias_init_with_prob$srcCsVdt|jjd}t|j| |t|dr'|jdur)t|j| |dSdSdS)a Initialize the weights and biases of a linear module. This function initializes the weights of a linear module using a uniform distribution within bounds calculated from the input dimension. If the module has a bias, it is also initialized. Args: module (nn.Module): Linear module to initialize. Returns: (nn.Module): The initialized module. Examples: >>> import torch.nn as nn >>> linear = nn.Linear(10, 5) >>> initialized_linear = linear_init(linear) rrbiasN)mathsqrtweightshaperhasattrr)r boundrrr linear_init:s r!h㈵>cCs8|jddd}|j|d}d|j|d}t||S)a Calculate the inverse sigmoid function for a tensor. This function applies the inverse of the sigmoid function to a tensor, which is useful in various neural network operations, particularly in attention mechanisms and coordinate transformations. Args: x (torch.Tensor): Input tensor with values in range [0, 1]. eps (float, optional): Small epsilon value to prevent numerical instability. Returns: (torch.Tensor): Tensor after applying the inverse sigmoid function. Examples: >>> x = torch.tensor([0.2, 0.5, 0.8]) >>> inverse_sigmoid(x) tensor([-1.3863, 0.0000, 1.3863]) rr)minmax)r#)clamptorchr)xepsx1x2rrr rRs rvaluevalue_spatial_shapessampling_locationsattention_weightsreturncCs*|j\}}}}|j\}}}} } }|jdd|Ddd} d|d} g} t|D]>\}\}}| |ddd|||||}| dddddd|fdddd}tj||dd d d }| |q(|dd||d|| | }t j | d dd | d  ||||}|dd S)a Implement multi-scale deformable attention in PyTorch. This function performs deformable attention across multiple feature map scales, allowing the model to attend to different spatial locations with learned offsets. Args: value (torch.Tensor): The value tensor with shape (bs, num_keys, num_heads, embed_dims). value_spatial_shapes (torch.Tensor): Spatial shapes of the value tensor with shape (num_levels, 2). sampling_locations (torch.Tensor): The sampling locations with shape (bs, num_queries, num_heads, num_levels, num_points, 2). attention_weights (torch.Tensor): The attention weights with shape (bs, num_queries, num_heads, num_levels, num_points). Returns: (torch.Tensor): The output tensor with shape (bs, num_queries, embed_dims). References: https://github.com/IDEA-Research/detrex/blob/main/detrex/layers/multi_scale_deform_attn.py cSsg|]\}}||qSrr)rH_W_rrr r sz7multi_scale_deformable_attn_pytorch..r)dimNrbilinearzerosF)mode padding_mode align_corners)rsplit enumerateflatten transposereshapeF grid_sampleappendr&stacksumview contiguous)r+r,r-r.bsr num_heads embed_dims num_queries num_levels num_points value_listsampling_gridssampling_value_listlevelr0r1value_l_sampling_grid_l_sampling_value_l_outputrrr rks( &,   r)r)r")rrnumpyrr&torch.nnrtorch.nn.functional functionalr@ torch.nn.initr__all__rrr!rTensorrrrrr s.