o H h7)@sxUddlZddlZddlZddlmZmZmZmZddlZddl ZddZ ddZ ddZ d d Z d'd d ZddZddddZejje ejje eje ejjjeejje ejeejeiZeeefed<ddZGdddejjZGdddeZ Gddde Z!ddZ"Gdddejj#Z$  d(d eejj%ed!effd"eee&ej'fd#eee&efd$ejj(fd%d&Z)dS))N)AnyCallableOptionalUnioncCs$tjg|j|jjdRddiS)Ndevicemeta)torchemptyshapeweightselfinputrU/var/www/vscode/kcb/lib/python3.10/site-packages/torch/fx/experimental/meta_tracer.pyembedding_override s$rcC|SNrr rrrnn_layernorm_overridercCs|Srr)xrrrtorch_relu_overriderrcCrrr)rrrrrtorch_nn_relu_overriderrFcCs|rJd|S)Nz.wrapper..check_has_proxy call_function)r r%node map_aggregatetracer create_proxy)argskwargsr+targetr)rwrapper7s z(gen_constructor_wrapper..wrapper) functoolswraps)r4r5rr3rgen_constructor_wrapper6sr8c@sReZdZddZdddZddZedd Zed d Zed d Z ddZ dS) MetaProxycCs ||_dSr) _tensor_meta)r tensor_metarrrinstall_tensor_metaL zMetaProxy.install_tensor_metaNcCsPt|dr|jdur|jj|r|gSgS|jdd|r#||fiS|fiS)Nr: call_methodsize)hasattrr:r?r/r0)rdimrrrr?OszMetaProxy.sizecCs2t|dr|jdur|jS|jdd|fiS)Nr:r>rA)r@r:rAr/r0rrrrrAVs z MetaProxy.dimcC4t|dr|jdur|jjS|jdtj|dfiS)Nr:r,r )r@r:r r/r0builtinsgetattrrBrrrr [ zMetaProxy.shapecCrC)Nr:r,dtype)r@r:rGr/r0rDrErBrrrrGcrFzMetaProxy.dtypecCs t|dS)Nr)MetaDeviceAttributerBrrrrks zMetaProxy.devicecCs|dkr ||St||S)Nr:)__getattribute__ MetaAttribute)rkrrr __getattr__qs  zMetaProxy.__getattr__r) __name__ __module__ __qualname__r<r?rApropertyr rGrrLrrrrr9Ks     r9c@s.eZdZdefddZeddZddZdS) rJattrcCs||_||_|j|_d|_dSr)rootrQr/_node)rrRrQrrr__init__zs zMetaAttribute.__init__cCs.|jdur|jdt|j|jfij|_|jS)Nr,)rSr/r0rErRrQr-rBrrrr-s zMetaAttribute.nodecOs|jd|j|jf||S)Nr>)r/r0rQrR)rr1r2rrr__call__szMetaAttribute.__call__N)rMrNrOstrrTrPr-rUrrrrrJys   rJc@s eZdZdS)rHN)rMrNrOrrrrrHsrHcCsTt|trdSt|tjjr(t|tsJdt|t|ds%Jd|jS|S)NrzExpected MetaProxy but got r:z*MetaProxy does not have an associated meta) r$rHr r%r&r9typer@r:r'rrrproxys_to_metass rXcseZdZUdZeed<gdZ   dfdd ZfddZfd d Z d e j j d e fd dZd e j j d e ffdd ZddZddee e jfffdd ZZS) MetaTracerTallow_insert_stateless_mods)arangezerosones full_likeeyeNc st|||||||}|dkr||jvr||j||S||jvr,d|vr,d|d<ztjj|t } tjj|t } |dkrOt ||} | | i| } n|dkrft | d|} | | ddi| } np|dkrt |d sqJd |_z)|j|} t| }|t vrt || g| Ri| } n|j| i| } Wd |_n8d |_w|d krd |_z%|j}|d }|D]}t ||}qt|tjsJ|jdd} Wd |_nd |_w|WSt|tjjsJd|| W|Sty }ztd|d|d|WYd}~|Sd}~ww)N placeholderrrr,r>r call_module orig_forwardTFget_attr.rz!Dont support composite output yetzCould not compute metadata for z target z: )superr0 meta_argsr<orig_fnsr r%r-r.rXr#getrEr@_disable_module_getattrrR get_submodulerWrcsplitr$Tensorrr& Exceptionwarningswarn)rkindr4r1r2name type_exprproxy_factory_fnrv args_metas kwargs_metas meta_targetmeta_outmodmod_typeattr_itratomsatome __class__rrr0sl       &zMetaTracer.create_proxycs t|ddr|St|||S)NrjF)rErf)rrQattr_valparameter_proxy_cacherrrrEs zMetaTracer.getattrcs||_t||||Sr)rcrfrb)rmforwardr1r2rrrrbszMetaTracer.call_modulerzreturncCs^d}|jj}|d|}t|j|r&|d|}|d7}t|j|s|j|||S)zb Helper method which tries to insert a module that was not declared as submodule. r_ra)rrMlowerr@rR add_module)rrzidxmod_namepathrrr_insert_module_as_submodules   z&MetaTracer._insert_module_as_submodulecsfzt|WSty2|jr1tt|dkr1tt|dkr1||}||_ |YSw)Nr) rfpath_of_module NameErrorrZlenlist parametersbuffersr prev_module)rrzrrrrrs  zMetaTracer.path_of_modulecCs t||Sr)r9)rr-rrrr*r=zMetaTracer.proxyrgc st|tsJ||_dd|jD|_t|_|jD]\}\}}tt |||j |qz!t ||}d|i|_ |W|jD] \}\}}tt ||qCS|jD] \}\}}tt ||qVw)NcSsi|] }|ttt|qSr)r8rEr ).0r4rrr sz$MetaTracer.trace..rg)r$dictrg_TORCH_METHODS_TO_PATCHpatched_torch_methodssetrhitemssetattrr addrftrace_tracer_extras) rrRrg concrete_argsrrr5origgraphrrrrrs$  zMetaTracer.trace)NNNr)rMrNrOrZbool__annotations__rr0rErbr nnModulerVrrr*rrmr __classcell__rrrrrYs  I &rYrR.rgrrcCsFt}||||}t|tjjr|jjn|j}tj |j ||}|Sr) rYrr$r rrrrMr% GraphModulerR)rRrgrr/rrrgmrrrsymbolic_trace,s r)F)NN)*rDr6rotypingrrrrr torch.fxrrrrrr r"r Embedding LayerNormrelu functionalReLUwhereabsr#rrr8r%r&r9rJrHrXTracerrYrrVrmrrrrrrsN    .