o W h@sddlmZmZmZmZddlZddlmZddlmm Z ddl m Z ddl mZmZmZmZmZmZmZmZGdddejZGdd d ejZGd d d ejZGd d d ejZGdddejZGdddejZdS))ListOptionalTupleTypeN) LayerNorm2d)BlockCXBlockFuserMaskDownSamplerMultiScaleBlock PatchEmbedPositionEmbeddingRandomPositionEmbeddingSinec#seZdZdZddddddddd ejejd d d d d fd edededededededede de ej de ej de de de dede edfddf"fd d! Z d"ejdejfd#d$ZZS)%ImageEncoderViTa1 An image encoder using Vision Transformer (ViT) architecture for encoding images into a compact latent space. This class processes images by splitting them into patches, applying transformer blocks, and generating a final encoded representation through a neck module. Attributes: img_size (int): Dimension of input images, assumed to be square. patch_embed (PatchEmbed): Module for patch embedding. pos_embed (nn.Parameter | None): Absolute positional embedding for patches. blocks (nn.ModuleList): List of transformer blocks for processing patch embeddings. neck (nn.Sequential): Neck module to further process the output. Methods: forward: Processes input through patch embedding, positional embedding, blocks, and neck. Examples: >>> import torch >>> encoder = ImageEncoderViT(img_size=224, patch_size=16, embed_dim=768, depth=12, num_heads=12) >>> input_image = torch.randn(1, 3, 224, 224) >>> output = encoder(input_image) >>> print(output.shape) i g@TFrimg_size patch_sizein_chans embed_dimdepth num_heads mlp_ratio out_chansqkv_bias norm_layer act_layer use_abs_pos use_rel_posrel_pos_zero_init window_sizeglobal_attn_indexes.returnNcst||_t||f||f||d|_d|_| r*tt d||||||_t |_ t |D]"}t |||| | | | |||vrD|nd||||fd }|j |q3ttj||dddt|tj||dddd t||_dS) a[ Initialize an ImageEncoderViT instance for encoding images using Vision Transformer architecture. Args: img_size (int): Input image size, assumed to be square. patch_size (int): Size of image patches. in_chans (int): Number of input image channels. embed_dim (int): Dimension of patch embeddings. depth (int): Number of transformer blocks. num_heads (int): Number of attention heads in each block. mlp_ratio (float): Ratio of MLP hidden dimension to embedding dimension. out_chans (int): Number of output channels from the neck module. qkv_bias (bool): If True, adds learnable bias to query, key, value projections. norm_layer (Type[nn.Module]): Type of normalization layer to use. act_layer (Type[nn.Module]): Type of activation layer to use. use_abs_pos (bool): If True, uses absolute positional embeddings. use_rel_pos (bool): If True, adds relative positional embeddings to attention maps. rel_pos_zero_init (bool): If True, initializes relative positional parameters to zero. window_size (int): Size of attention window for windowed attention blocks. global_attn_indexes (Tuple[int, ...]): Indices of blocks that use global attention. Examples: >>> encoder = ImageEncoderViT(img_size=224, patch_size=16, embed_dim=768, depth=12, num_heads=12) >>> input_image = torch.randn(1, 3, 224, 224) >>> output = encoder(input_image) >>> print(output.shape) ) kernel_sizestriderrNrr) dimrrrr r!r#r$r% input_sizeF)r(biasr)r(paddingr,)super__init__rr patch_embed pos_embednn Parametertorchzeros ModuleListblocksrangerappend SequentialConv2drneck)selfrrrrrrrrrr r!r"r#r$r%r&iblock __class__r[/var/www/vscode/kcb/lib/python3.10/site-packages/ultralytics/models/sam/modules/encoders.pyr/0sV .     zImageEncoderViT.__init__xcCs||}|jdur-|jdkr&tj|jdddd|jddddddn|j}||}|jD]}||}q0||ddddS)zaProcess input through patch embedding, positional embedding, transformer blocks, and neck module.Nrrrr) scale_factor)r0r1rF interpolatepermuter7r<)r=rCr1blkrrrBforwards   .  zImageEncoderViT.forward)__name__ __module__ __qualname____doc__r2 LayerNormGELUintfloatboolrModulerr/r4TensorrJ __classcell__rrr@rBrsl      _rc s>eZdZdZejfdedeeefdeeefdedeej ddf fd d Z de j fd d Z d e j de j dede j fddZde j de j fddZde j de j fddZed eee j e j fdee j dee j defddZde jfddZd eee j e j fdee j dee j dee j e j ffddZZS) PromptEncodera: Encodes different types of prompts for input to SAM's mask decoder, producing sparse and dense embeddings. Attributes: embed_dim (int): Dimension of the embeddings. input_image_size (Tuple[int, int]): Size of the input image as (H, W). image_embedding_size (Tuple[int, int]): Spatial size of the image embedding as (H, W). pe_layer (PositionEmbeddingRandom): Module for random position embedding. num_point_embeddings (int): Number of point embeddings for different types of points. point_embeddings (nn.ModuleList): List of point embeddings. not_a_point_embed (nn.Embedding): Embedding for points that are not part of any label. mask_input_size (Tuple[int, int]): Size of the input mask. mask_downscaling (nn.Sequential): Neural network for downscaling the mask. no_mask_embed (nn.Embedding): Embedding for cases where no mask is provided. Methods: get_dense_pe: Returns the positional encoding used to encode point prompts. forward: Embeds different types of prompts, returning both sparse and dense embeddings. Examples: >>> prompt_encoder = PromptEncoder(256, (64, 64), (1024, 1024), 16) >>> points = (torch.rand(1, 5, 2), torch.randint(0, 4, (1, 5))) >>> boxes = torch.rand(1, 2, 2) >>> masks = torch.rand(1, 1, 256, 256) >>> sparse_embeddings, dense_embeddings = prompt_encoder(points, boxes, masks) >>> print(sparse_embeddings.shape, dense_embeddings.shape) torch.Size([1, 7, 256]) torch.Size([1, 256, 64, 64]) rimage_embedding_sizeinput_image_size mask_in_chans activationr'Nc st|_||_||_td|_d|_fddt|jD}t ||_ t d|_ d|dd|df|_t t jd|ddddt|d|t j|d|dddt||t j|dd|_t d|_d S) a! Initialize the PromptEncoder module for encoding various types of prompts. Args: embed_dim (int): The dimension of the embeddings. image_embedding_size (Tuple[int, int]): The spatial size of the image embedding as (H, W). input_image_size (Tuple[int, int]): The padded size of the input image as (H, W). mask_in_chans (int): The number of hidden channels used for encoding input masks. activation (Type[nn.Module]): The activation function to use when encoding input masks. Examples: >>> prompt_encoder = PromptEncoder(256, (64, 64), (1024, 1024), 16) >>> points = (torch.rand(1, 5, 2), torch.randint(0, 4, (1, 5))) >>> boxes = torch.rand(1, 2, 2) >>> masks = torch.rand(1, 1, 256, 256) >>> sparse_embeddings, dense_embeddings = prompt_encoder(points, boxes, masks) >>> print(sparse_embeddings.shape, dense_embeddings.shape) torch.Size([1, 7, 256]) torch.Size([1, 256, 64, 64]) rDcsg|]}tdqSr)r2 Embedding).0_rrrB z*PromptEncoder.__init__..rr)r(r)r(N)r.r/rrYrXrpe_layernum_point_embeddingsr8r2r6point_embeddingsr^not_a_point_embedmask_input_sizer:r;rmask_downscaling no_mask_embed)r=rrXrYrZr[rgr@rarBr/s(    zPromptEncoder.__init__cCs||jdS)a Return the dense positional encoding used for encoding point prompts. Generate a positional encoding for a dense set of points matching the shape of the image encoding. The encoding is used to provide spatial information to the model when processing point prompts. Returns: (torch.Tensor): Positional encoding tensor with shape (1, embed_dim, H, W), where H and W are the height and width of the image embedding size, respectively. Examples: >>> prompt_encoder = PromptEncoder(256, (64, 64), (1024, 1024), 16) >>> dense_pe = prompt_encoder.get_dense_pe() >>> print(dense_pe.shape) torch.Size([1, 256, 64, 64]) r)rerX unsqueezer=rrrB get_dense_peszPromptEncoder.get_dense_pepointslabelspadcCs|d}|r4tj|jdddf|jd}tj|jddf|jd }tj||gdd}tj||gdd}|j||j}d||dk<||dk|j j 7<||dk|j dj 7<||dk|j dj 7<||dk|j dj 7<||d k|j d j 7<|S) zREmbed point prompts by applying positional encoding and label-specific embeddings.?rrrDdevicer*r) r4r5shapertonescatreforward_with_coordsrYrhweightrg)r=rorprq padding_point padding_labelpoint_embeddingrrrB _embed_pointss zPromptEncoder._embed_pointsboxescCsv|d}|ddd}|j||j}|dddddf|jdj7<|dddddf|jdj7<|S)zOEmbed box prompts by applying positional encoding and adding corner embeddings.rrrwrDNrrr)reshaperer{rYrgr|)r=rcoordscorner_embeddingrrrB _embed_boxess &&zPromptEncoder._embed_boxesmaskscCs ||S)zMEmbed mask inputs by downscaling and processing through convolutional layers.)rj)r=rrrrB _embed_maskss zPromptEncoder._embed_maskscCs>|dur |djdS|dur|jdS|dur|jdSdS)zKGet the batch size of the output given the batch size of the input prompts.Nrr)rx)rorrrrrB_get_batch_sizes  zPromptEncoder._get_batch_sizecCs|jdjjS)z?Return the device of the first point embedding's weight tensor.r)rgr|rtrmrrrB _get_device/szPromptEncoder._get_devicec Cs||||}tj|d|jf|d}|dur/|\}}|j|||dud}tj||gdd}|durA||} tj|| gdd}|durN||} || fS|j j dddd |d|j d|j d} || fS)a> Embed different types of prompts, returning both sparse and dense embeddings. Args: points (Tuple[torch.Tensor, torch.Tensor] | None): Point coordinates and labels to embed. The first tensor contains coordinates with shape (B, N, 2), and the second tensor contains labels with shape (B, N). boxes (torch.Tensor | None): Boxes to embed with shape (B, M, 2, 2), where M is the number of boxes. masks (torch.Tensor | None): Masks to embed with shape (B, 1, H, W). Returns: (Tuple[torch.Tensor, torch.Tensor]): A tuple containing: - sparse_embeddings (torch.Tensor): Sparse embeddings for points and boxes with shape (B, N, embed_dim). - dense_embeddings (torch.Tensor): Dense embeddings for masks of shape (B, embed_dim, embed_H, embed_W). Examples: >>> encoder = PromptEncoder(256, (64, 64), (1024, 1024), 16) >>> points = (torch.rand(1, 5, 2), torch.randint(0, 4, (1, 5))) >>> boxes = torch.rand(1, 2, 2, 2) >>> masks = torch.rand(1, 1, 256, 256) >>> sparse_emb, dense_emb = encoder(points, boxes, masks) >>> print(sparse_emb.shape, dense_emb.shape) torch.Size([1, 7, 256]) torch.Size([1, 256, 64, 64]) rrsN)rqrrurw)rr4emptyrrrrzrrrkr|rexpandrX) r=rorrbssparse_embeddingsrrprgbox_embeddingsdense_embeddingsrrrBrJ3s   zPromptEncoder.forward)rKrLrMrNr2rPrQrrrTr/r4rUrnrSrrr staticmethodrrrtrrJrVrrr@rBrWsP#  2   rWc sReZdZdZ d fdd Z d dejdejded eejejffd d Z Z S) MemoryEncodera Encode pixel features and masks into a memory representation for efficient image segmentation. This class processes pixel-level features and masks, fusing them to generate encoded memory representations suitable for downstream tasks in image segmentation models like SAM (Segment Anything Model). Attributes: mask_downsampler (MaskDownSampler): Module for downsampling input masks. pix_feat_proj (nn.Conv2d): Convolutional layer for projecting pixel features. fuser (Fuser): Module for fusing pixel features and masks. position_encoding (PositionEmbeddingSine): Module for adding positional encoding to features. out_proj (nn.Module): Output projection layer, either nn.Identity or nn.Conv2d. Methods: forward: Process input pixel features and masks to generate encoded memory representations. Examples: >>> import torch >>> encoder = MemoryEncoder(out_dim=256, in_dim=256) >>> pix_feat = torch.randn(1, 256, 64, 64) >>> masks = torch.randn(1, 1, 64, 64) >>> encoded_feat, pos = encoder(pix_feat, masks) >>> print(encoded_feat.shape, pos.shape) torch.Size([1, 256, 64, 64]) torch.Size([1, 128, 64, 64]) rcsxttdddd|_tj||dd|_ttdddd|_ t d d |_ t |_ ||kr:tj||dd|_ d Sd S) ah Initialize the MemoryEncoder for encoding pixel features and masks into memory representations. This encoder processes pixel-level features and masks, fusing them to generate encoded memory representations suitable for downstream tasks in image segmentation models like SAM (Segment Anything Model). Args: out_dim (int): Output dimension of the encoded features. in_dim (int): Input dimension of the pixel features. Default is 256. Examples: >>> encoder = MemoryEncoder(out_dim=256, in_dim=256) >>> pix_feat = torch.randn(1, 256, 64, 64) >>> masks = torch.randn(1, 1, 64, 64) >>> encoded_feat, pos = encoder(pix_feat, masks) >>> print(encoded_feat.shape, pos.shape) torch.Size([1, 256, 64, 64]) torch.Size([1, 128, 64, 64]) rrDr)r(r)r-rdrru) num_layers@ num_pos_featsN)r.r/r mask_downsamplerr2r; pix_feat_projr r fuserrposition_encodingIdentityout_proj)r=out_dimin_dimr@rrBr/s   zMemoryEncoder.__init__Fpix_featrskip_mask_sigmoidr'cCsh|st|}||}||j}||}||}||}||}|||j }||gdS)z]Process pixel features and masks to generate encoded memory representations for segmentation.)vision_featuresvision_pos_enc) rFsigmoidrtortrrrrdtype)r=rrrrCposrrrBrJs       zMemoryEncoder.forward)r)F) rKrLrMrNr/r4rUrSrrJrVrrr@rBres&rcsFeZdZdZ d dejdejdeffdd Zdej fd d Z Z S) ImageEncodera Encode images using a trunk-neck architecture, producing multiscale features and positional encodings. This class combines a trunk network for feature extraction with a neck network for feature refinement and positional encoding generation. It can optionally discard the lowest resolution features. Attributes: trunk (nn.Module): The trunk network for initial feature extraction. neck (nn.Module): The neck network for feature refinement and positional encoding generation. scalp (int): Number of lowest resolution feature levels to discard. Methods: forward: Process the input image through the trunk and neck networks. Examples: >>> trunk = SomeTrunkNetwork() >>> neck = SomeNeckNetwork() >>> encoder = ImageEncoder(trunk, neck, scalp=1) >>> image = torch.randn(1, 3, 224, 224) >>> output = encoder(image) >>> print(output.keys()) dict_keys(['vision_features', 'vision_pos_enc', 'backbone_fpn']) rtrunkr<scalpcsNt||_||_||_|jj|jjks%Jd|jjd|jjddS)a Initialize the ImageEncoder with trunk and neck networks for feature extraction and refinement. This encoder combines a trunk network for feature extraction with a neck network for feature refinement and positional encoding generation. It can optionally discard the lowest resolution features. Args: trunk (nn.Module): The trunk network for initial feature extraction. neck (nn.Module): The neck network for feature refinement and positional encoding generation. scalp (int): Number of lowest resolution feature levels to discard. Examples: >>> trunk = SomeTrunkNetwork() >>> neck = SomeNeckNetwork() >>> encoder = ImageEncoder(trunk, neck, scalp=1) >>> image = torch.randn(1, 3, 224, 224) >>> output = encoder(image) >>> print(output.keys()) dict_keys(['vision_features', 'vision_pos_enc', 'backbone_fpn']) zChannel dims of trunk z and neck z do not match.N)r.r/rr<r channel_listbackbone_channel_list)r=rr<rr@rrBr/s zImageEncoder.__init__samplecCsT|||\}}|jdkr |d|j |d|j }}|d}|||dS)z`Encode input through patch embedding, positional embedding, transformer blocks, and neck module.rNrw)rr backbone_fpn)r<rr)r=rfeaturesrsrcrrrBrJs "zImageEncoder.forward)r) rKrLrMrNr2rTrQr/r4rUrJrVrrr@rBrs"rcspeZdZdZ      ddedeed ed ed ed ed edeeeffdd Zdee j fddZ Z S)FpnNecka A Feature Pyramid Network (FPN) neck variant for multiscale feature fusion in object detection models. This FPN variant removes the output convolution and uses bicubic interpolation for feature resizing, similar to ViT positional embedding interpolation. Attributes: position_encoding (PositionEmbeddingSine): Sinusoidal positional encoding module. convs (nn.ModuleList): List of convolutional layers for each backbone level. backbone_channel_list (List[int]): List of channel dimensions from the backbone. fpn_interp_model (str): Interpolation mode for FPN feature resizing. fuse_type (str): Type of feature fusion, either 'sum' or 'avg'. fpn_top_down_levels (List[int]): Levels to have top-down features in outputs. Methods: forward: Perform forward pass through the FPN neck. Examples: >>> backbone_channels = [64, 128, 256, 512] >>> fpn_neck = FpnNeck(256, backbone_channels) >>> inputs = [torch.rand(1, c, 32, 32) for c in backbone_channels] >>> outputs, positions = fpn_neck(inputs) >>> print(len(outputs), len(positions)) 4 4 rrbilinearsumNd_modelrr(r)r-fpn_interp_model fuse_typefpn_top_down_levelsc sttdd|_t|_||_|D]} t} | dtj | ||||d|j | q||_ |dvs9J||_ |durGtt|j}t||_dS)a Initializes a modified Feature Pyramid Network (FPN) neck. This FPN variant removes the output convolution and uses bicubic interpolation for feature resizing, similar to ViT positional embedding interpolation. Args: d_model (int): Dimension of the model. backbone_channel_list (List[int]): List of channel dimensions from the backbone. kernel_size (int): Kernel size for the convolutional layers. stride (int): Stride for the convolutional layers. padding (int): Padding for the convolutional layers. fpn_interp_model (str): Interpolation mode for FPN feature resizing. fuse_type (str): Type of feature fusion, either 'sum' or 'avg'. fpn_top_down_levels (Optional[List[int]]): Levels to have top-down features in outputs. Examples: >>> backbone_channels = [64, 128, 256, 512] >>> fpn_neck = FpnNeck(256, backbone_channels) >>> print(fpn_neck) rrconv) in_channels out_channelsr(r)r->avgrN)r.r/rrr2r6convsrr: add_moduler;r9rrr8lenlistr) r=rrr(r)r-rrrr*currentr@rrBr/s.    zFpnNeck.__init__xsc Csdgt|j}dgt|j}t|t|jksJd}t|jd}t|ddD]P}||}|j|||}||jvrg|durgtj|jtjdd|j |j dkrTdnddd} || }|j d krf|d }n|}|} | ||<| | | j ||<q*||fS) aZ Performs forward pass through the Feature Pyramid Network (FPN) neck. This method processes a list of input tensors from the backbone through the FPN, applying lateral connections and top-down feature fusion. It generates output feature maps and corresponding positional encodings. Args: xs (List[torch.Tensor]): List of input tensors from the backbone, each with shape (B, C, H, W). Returns: (Tuple[List[torch.Tensor], List[torch.Tensor]]): A tuple containing: - out (List[torch.Tensor]): List of output feature maps after FPN processing, each with shape (B, d_model, H, W). - pos (List[torch.Tensor]): List of positional encodings corresponding to each output feature map. Examples: >>> fpn_neck = FpnNeck(d_model=256, backbone_channel_list=[64, 128, 256, 512]) >>> inputs = [torch.rand(1, c, 32, 32) for c in [64, 128, 256, 512]] >>> outputs, positions = fpn_neck(inputs) >>> print(len(outputs), len(positions)) 4 4 Nrrw)r@nearestF)rEmode align_corners antialiasrrD) rrr8rrFrGrr4float32rrrr) r=routr prev_featuresnr>rClateral_featurestop_down_featuresx_outrrrBrJ^s2  zFpnNeck.forward)rrrrrN) rKrLrMrNrQrstrrr/r4rUrJrVrrr@rBrs4 ?rcseZdZdZ         d"d ededededeeefdeedfdededeeefdeedfdeedfffdd Zdeeefdej fddZ dej de ej fd d!Z Z S)#Hieraa Hierarchical vision transformer for efficient multiscale feature extraction in image processing tasks. This class implements a Hiera model, which is a hierarchical vision transformer architecture designed for efficient multiscale feature extraction. It uses a series of transformer blocks organized into stages, with optional pooling and global attention mechanisms. Attributes: window_spec (Tuple[int, ...]): Window sizes for each stage. q_stride (Tuple[int, int]): Downsampling stride between stages. stage_ends (List[int]): Indices of the last block in each stage. q_pool_blocks (List[int]): Indices of blocks where pooling is applied. return_interm_layers (bool): Whether to return intermediate layer outputs. patch_embed (PatchEmbed): Module for patch embedding. global_att_blocks (Tuple[int, ...]): Indices of blocks with global attention. window_pos_embed_bkg_spatial_size (Tuple[int, int]): Spatial size for window positional embedding background. pos_embed (nn.Parameter): Positional embedding for the background. pos_embed_window (nn.Parameter): Positional embedding for the window. blocks (nn.ModuleList): List of MultiScaleBlock modules. channel_list (List[int]): List of output channel dimensions for each stage. Methods: _get_pos_embed: Generate positional embeddings by interpolating and combining window and background embeddings. forward: Perform the forward pass through the Hiera model. Examples: >>> model = Hiera(embed_dim=96, num_heads=1, stages=(2, 3, 16, 3)) >>> input_tensor = torch.randn(1, 3, 224, 224) >>> output_features = model(input_tensor) >>> for feat in output_features: ... print(feat.shape) `rrvrrDrDrDrrrrrr\rrrTrrdrop_path_rateq_poolq_stridestages.dim_mulhead_mul!window_pos_embed_bkg_spatial_size window_specglobal_att_blocksc  s ttt| ksJ| _t} |_fddtdtdD_d|kr>> model = Hiera(embed_dim=96, num_heads=1, stages=(2, 3, 16, 3)) >>> input_tensor = torch.randn(1, 3, 224, 224) >>> output_features = model(input_tensor) >>> for feat in output_features: ... print(feat.shape) cs g|] }td|dqS)Nr)rr_r>)rrrBrbs z"Hiera.__init__..rrNrwcSsg|]}|dqSr]rr_rCrrrBrb)rr)r\r\)rr)rr(r)r-cSsg|]}|qSr)itemrrrrBrbr)r*dim_outr drop_pathrr%csg|]}j|jqSr)r7rrrmrrBrb'rc)r.r/rrrrr8 stage_ends q_pool_blocksreturn_interm_layersr r0rrr2r3r4r5r1pos_embed_windowlinspacer6r7rQr r9rr)r=rrrrrrrrrrrrrdpr cur_stager>rr%r?r@)r=rrBr/s\ 6"*"$       zHiera.__init__hwr'cCsZ|\}}|j}tj|j||fdd}||ddt|j|jD}|dddd}|S) z_Generate positional embeddings by interpolating and combining window and background embeddings.bicubic)sizercSsg|]\}}||qSrr)r_rCyrrrBrb1rcz(Hiera._get_pos_embed..rrDrr)rrFrGr1tileziprxrH)r=rhw window_embedr1rrrB_get_pos_embed,s "zHiera._get_pos_embedrCcCs~||}|||jdd}g}t|jD]$\}}||}||jdks/||jvr<|jr<|dddd}||q|S)a Perform forward pass through Hiera model, extracting multiscale features from input images. Args: x (torch.Tensor): Input tensor with shape (B, C, H, W) representing a batch of images. Returns: (List[torch.Tensor]): List of feature maps at different scales, each with shape (B, C_i, H_i, W_i), where C_i is the channel dimension and H_i, W_i are the spatial dimensions at scale i. The list is ordered from highest resolution (fine features) to lowest resolution (coarse features) if return_interm_layers is True, otherwise contains only the final output. Examples: >>> model = Hiera(embed_dim=96, num_heads=1, stages=(2, 3, 16, 3)) >>> input_tensor = torch.randn(1, 3, 224, 224) >>> output_features = model(input_tensor) >>> for feat in output_features: ... print(feat.shape) rrrwrrD) r0rrx enumerater7rrrHr9)r=rCoutputsr>rIfeatsrrrBrJ5s  z Hiera.forward) rrrvrrrrrrrrT)rKrLrMrNrQrRrr/r4rUrrrJrVrrr@rBrsN#      v" r)typingrrrrr4torch.nnr2torch.nn.functional functionalrFultralytics.nn.modulesrr7rr r r r r rrrTrrWrrrrrrrrBs  ( HUJ