o V h(l@sddlmZddlmZmZmZmZmZddlZddl m m Z ddlm Z m Z ddlmZddlmZdd lmZmZmZdd lmZdd lmZmZdd lmZmZgd ZdeedeedeedeeeeeffddZ ej!"ddej dej deede fddZ#ej!"ddeeeefdeeeefdeeeeffddZ$ej!"dde deeeefdeeeefdeeeefde f ddZ%ej!"d     dLd e d!e d"e d#e deed$edeed%e&d&e&d'ee d(ee d)e'de fd*d+Z(ej!"d+Gd,d-d-e j)Z*Gd.d/d/e j)Z+Gd0d1d1e j)Z,deed2ed3eed$eedeed4e&d5eed6e'd7ede,fd8d9Z-ed:d;d<Z.Gd=d>d>eZ/Gd?d@d@eZ0GdAdBdBeZ1eedCe/j2fdDdddEd5ee/d6e'd7ede,fdFdGZ3eedCe0j2fdDdddEd5ee0d6e'd7ede,fdHdIZ4eedCe1j2fdDdddEd5ee1d6e'd7ede,fdJdKZ5dS)M)partial)AnyCallableListOptionalTupleN)nnTensor)VideoClassification)_log_api_usage_once)register_modelWeights WeightsEnum)_KINETICS400_CATEGORIES)_ovewrite_named_paramhandle_legacy_interface) PatchMergingSwinTransformerBlock)SwinTransformer3dSwin3D_T_WeightsSwin3D_S_WeightsSwin3D_B_Weightsswin3d_tswin3d_sswin3d_b shift_sizesize_dhw window_sizereturncCs:tdD]}||||kr||||<d||<q||fS)Nr rrange)rrrir$]/var/www/vscode/kcb/lib/python3.10/site-packages/torchvision/models/video/swin_transformer.py_get_window_and_shift_size s  r&relative_position_bias_tablerelative_position_indexcCs^|d|d|d}||d|d|f}|||d}|dddd}|S)Nrr )flattenviewpermute contiguous unsqueeze)r'r(r window_volrelative_position_biasr$r$r%_get_relative_position_bias/sr2 patch_sizecs.fddtdD}|d|d|dfS)Ncs,g|]}||||qSr$r$.0r#r3rr$r% @s,z(_compute_pad_size_3d..r rr)r r!)rr3pad_sizer$r6r%_compute_pad_size_3d?sr9xc sj|j|}|dd|dd|dd}fddtdD}d}|dD]-}|dD]&} |dD]} |||d|d| d| d| d| df<|d7}q;q5q/||ddd|ddd|ddd}|dddddd|ddd}|d|d}||dktd |dktd }|S) Nrr)r cs:g|]}d| f| | f| dffqS)rNr$r4rrr$r%r7Ps  z._compute_attention_mask_3d..r gY) new_zerosr"r,r-reshaper/ masked_fillfloat) r:rrr attn_mask num_windowsslicescountdhwr$r;r%_compute_attention_mask_3dGs6 0    2 $rJr>Tinput qkv_weight proj_weightr1 num_headsattention_dropoutdropoutqkv_bias proj_biastrainingc  Cs|j\} } }}}t| ||f|d|d|df}t|ddd|dd|dd|df}|j\}}}}}|||f}t|dkrTtj||d |d |d fdd}|d|d|d|d|d|d}|| |d|d|d|d|d|d|d|d|d|}|ddddddd d  | ||d|d|d|}t ||| }| | d| dd|||ddddd}|d|d|d}}}|||d }| | d d }||}t|dkrDt||d|d|df|d|d|df|d|d|df}|| d|||| d| d}||dd}|d || d| d}tj|d d}tj||| d}| | dd | d| d|}t ||| }tj||| d}|| |d|d|d|d|d|d|d|d|d|}|ddddddd d  | ||||}t|dkrtj||d|d|dfdd}|ddd| d|d|ddf}|S)a Window based multi-head self attention (W-MSA) module with relative position bias. It supports both of shifted and non-shifted window. Args: input (Tensor[B, T, H, W, C]): The input tensor, 5-dimensions. qkv_weight (Tensor[in_dim, out_dim]): The weight tensor of query, key, value. proj_weight (Tensor[out_dim, out_dim]): The weight tensor of projection. relative_position_bias (Tensor): The learned relative position bias added to attention. window_size (List[int]): 3-dimensions window size, T, H, W . num_heads (int): Number of attention heads. shift_size (List[int]): Shift size for shifted window attention (T, H, W). attention_dropout (float): Dropout ratio of attention weight. Default: 0.0. dropout (float): Dropout ratio of output. Default: 0.0. qkv_bias (Tensor[out_dim], optional): The bias tensor of query, key, value. Default: None. proj_bias (Tensor[out_dim], optional): The bias tensor of projection. Default: None. training (bool, optional): Training flag used by the dropout parameters. Default: True. Returns: Tensor[B, T, H, W, C]: The output tensor after shifted window attention. rr)r )r)r r )shiftsdimsr r=r<gr*)dim)prSN)shaper9Fpadsumtorchrollr,r-r@linearsizematmul transposerJr/softmaxrPr.)rKrLrMr1rrNrrOrPrQrRrSbtrHrIcr8r:_tphpwp padded_sizerDqkvqkvattnrCr$r$r%shifted_window_attention_3dsst!"(  (. 0((& "*rscseZdZdZ    ddedeedeededed ed ed ed d ffdd ZdddZ dddZ deed e j fddZ de d e fddZZS)ShiftedWindowAttention3dz2 See :func:`shifted_window_attention_3d`. Tr>rYrrrNrQrRrOrPr Nc stt|dkst|dkrtd||_||_||_||_||_t j ||d|d|_ t j |||d|_ | |dS)Nr z.window_size and shift_size must be of length 2)bias)super__init__len ValueErrorrrrNrOrPrLinearrnproj#define_relative_position_bias_tabledefine_relative_position_index) selfrYrrrNrQrRrOrP __class__r$r%rws  z!ShiftedWindowAttention3d.__init__cCs^ttd|jddd|jddd|jdd|j|_tjj|jdddS)Nr rr){Gz?std) r Parameterr_zerosrrNr'init trunc_normal_r~r$r$r%r|s4z.r rr)r ij)indexingr*r() r"r_stackmeshgridr+r-r.rr^register_buffer)r~ coords_dhwcoordscoords_flattenrelative_coordsr(r$rr%r}s ,(((>, z7ShiftedWindowAttention3d.define_relative_position_indexcCst|j|j|S)N)r2r'r()r~rr$r$r%get_relative_position_bias#sz3ShiftedWindowAttention3d.get_relative_position_biasr:c Cs|j\}}}}}|||g}|j|j}}t|||\}}||} t||jj|j j| ||j ||j |j |jj |j j |jd S)N)rrOrPrQrRrS)r[rcopyrr&rrsrnweightr{rNrOrPrurS) r~r:rirgrHrIrrrr1r$r$r%forward&s&  z ShiftedWindowAttention3d.forward)TTr>r>)r N)__name__ __module__ __qualname____doc__intrboolrBrwr|r}r_r rr __classcell__r$r$rr%rts:     rtc sbeZdZdZ   ddeedededeed ej fd df fd d Z d e d e fddZ Z S) PatchEmbed3da;Video to Patch Embedding. Args: patch_size (List[int]): Patch token size. in_channels (int): Number of input channels. Default: 3 embed_dim (int): Number of linear projection output channels. Default: 96. norm_layer (nn.Module, optional): Normalization layer. Default: None r `Nr3 in_channels embed_dim norm_layer.r csftt||d|d|df|_tj|||j|jd|_|dur,|||_dSt|_dS)Nrr)r ) kernel_sizestride) rvrwr tuple_patch_sizerConv3dr{normIdentity)r~r3rrrrr$r%rwKs zPatchEmbed3d.__init__r:c Cs||\}}}}}t|||f|j}t|d|dd|dd|df}||}|ddddd}|jdur<||}|S)zForward function.rr r)r r<N)rbr9rr\r]r{r-r)r~r:rirgrHrIr8r$r$r%ras$   zPatchEmbed3d.forward)r rN)rrrrrrrrrModulerwr rrr$r$rr%rAs" rc seZdZdZdddddddedf deeded eed eed eed ed ededededee de j fdee de j fde de j fdee de j fddffdd Z de de fddZZS)raY Implements 3D Swin Transformer from the `"Video Swin Transformer" `_ paper. Args: patch_size (List[int]): Patch size. embed_dim (int): Patch embedding dimension. depths (List(int)): Depth of each Swin Transformer layer. num_heads (List(int)): Number of attention heads in different layers. window_size (List[int]): Window size. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. Default: 4.0. dropout (float): Dropout rate. Default: 0.0. attention_dropout (float): Attention dropout rate. Default: 0.0. stochastic_depth_prob (float): Stochastic depth rate. Default: 0.1. num_classes (int): Number of classes for classification head. Default: 400. norm_layer (nn.Module, optional): Normalization layer. Default: None. block (nn.Module, optional): SwinTransformer Block. Default: None. downsample_layer (nn.Module): Downsample layer (patch merging). Default: PatchMerging. patch_embed (nn.Module, optional): Patch Embedding layer. Default: None. g@r>皙?iNr3rdepthsrNr mlp_ratiorPrOstochastic_depth_prob num_classesr.blockdownsample_layer patch_embedr cstt|| |_| durtttd} | dur!ttjdd} |dur't }|||| d|_ tj |d|_ g}t |}d}tt|D]S}g}|d|}t||D]*| t||d}|| ||||fd d |D||||| td |d7}qT|tj||t|dkr|| || qDtj||_|dt|d|_| |j|_td|_t|j| |_|D]}t|tjrtjj|jd d |jdurtj |jqdS)N) attn_layergh㈵>)eps)r3rr)rZrr r)cs$g|]}ddkr dn|dqS)r rr$)r5rIi_layerr$r%r7s$z.SwinTransformer3d.__init__..)rrrrPrOrrrrr)!rvrwr rrrrtr LayerNormrrDropoutpos_dropr^r"rxrBappend Sequentialfeatures num_featuresrAdaptiveAvgPool3davgpoolrzheadmodules isinstancerrrruzeros_)r~r3rrrNrrrPrOrrrrrrlayerstotal_stage_blocksstage_block_idi_stagestagerYsd_probmrrr%rwsd          zSwinTransformer3d.__init__r:cCs^||}||}||}||}|ddddd}||}t|d}||}|S)Nrr<r)r r ) rrrrr-rr_r+r)r~r:r$r$r%rs       zSwinTransformer3d.forward)rrrrrrrrBrrrrrwr rrr$r$rr%rnsV      Lrrrrweightsprogresskwargsc KsZ|durt|dt|jdtd||||||d|} |dur+| |j|dd| S)Nr categories)r3rrrNrrT)r check_hashr$)rrxmetarload_state_dictget_state_dict) r3rrrNrrrrrmodelr$r$r%_swin_transformer3ds  r)r)r)r))rmin_sizemin_temporal_sizec @LeZdZedeedddddieddd d d d d iddddZeZdS)rz9https://download.pytorch.org/models/swin3d_t-7615ae03.pthrg ףp= ?gv/?gCl?gZd;O?gy&1?g? crop_size resize_sizemeanrFhttps://github.com/SwinTransformer/Video-Swin-Transformer#kinetics-400The weights were ported from the paper. The accuracies are estimated on video-level with parameters `frame_rate=15`, `clips_per_video=12`, and `clip_len=32`iv Kinetics-400g(\mS@gK7aW@zacc@1zacc@5g7A`E@gnb^@recipe_docs num_params_metrics_ops _file_sizeurl transformsrN rrrrrr _COMMON_METAKINETICS400_V1DEFAULTr$r$r$r%r4rc @r)rz9https://download.pytorch.org/models/swin3d_s-da41c237.pthrrrrrrrif$rgMbXS@g'1W@rgҵT@gK7Ik@rrNrr$r$r$r%rrrc @seZdZedeedddddieddd d d d d iddddZedeedddddieddd d ddd iddddZeZ dS)rzsdr pretrained)r)rrc K:t|}td gddgdgdgdd||d|S) a Constructs a swin_tiny architecture from `Video Swin Transformer `_. Args: weights (:class:`~torchvision.models.video.Swin3D_T_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.video.Swin3D_T_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.video.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.video.Swin3D_T_Weights :members: r r<r<r)r r rVr r rV rWrWrr3rrrNrrrrNr$)rverifyrrrrr$r$r%rx  rc Kr) a Constructs a swin_small architecture from `Video Swin Transformer `_. Args: weights (:class:`~torchvision.models.video.Swin3D_S_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.video.Swin3D_S_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.video.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.video.Swin3D_S_Weights :members: rrr r r rrrrNr$)rrrrr$r$r%rrrc Kr) a Constructs a swin_base architecture from `Video Swin Transformer `_. Args: weights (:class:`~torchvision.models.video.Swin3D_B_Weights`, optional): The pretrained weights to use. See :class:`~torchvision.models.video.Swin3D_B_Weights` below for more details, and possible values. By default, no pre-trained weights are used. progress (bool, optional): If True, displays a progress bar of the download to stderr. Default is True. **kwargs: parameters passed to the ``torchvision.models.video.swin_transformer.SwinTransformer`` base class. Please refer to the `source code `_ for more details about this class. .. autoclass:: torchvision.models.video.Swin3D_B_Weights :members: rr)r<r rrrNr$)rrrrr$r$r%rrr)r>r>NNT)6 functoolsrtypingrrrrrr_torch.nn.functionalr functionalr\r transforms._presetsr utilsr _apirrr_metar_utilsrrswin_transformerrr__all__rr&fxwrapr2r9rJrBrrsrrtrrrrrrrrrrrr$r$r$r%s       4     )      p[-m  :*$*$.