o H h<0 @sddlZddlmZmZmZddlZddlmZddl m Z gdZ Gdddej j Zdej j d efd d Zd ej j d edefddZ  ddeejjej j fdeeej jgefdedefddZdS)N)CallableOptionalUnion)map_arg) split_module)FoldedGraphModuleget_unique_attr_name_in_modulesplit_const_subgraphsc sfeZdZdZ   ddejjdejjde ejjde e de f fd d Z fd d Z d dZ ZS)ra FoldedGraphModule is a GraphModule which also contains another `const_subgraph_module` representing a subgraph which has all const attr inputs and which can be run once before running the main standard `graph`. The `const_output_names` are the ordered list names of attrs which represent what each respective output from the const_subgraph should be set on which attrs. Ncudarootgraphconst_subgraphfx_const_folded_attrs_namedevice_for_folded_attrscs@t|||dur dntj|||_d|_||_||_dS)NF) super__init__torchfx GraphModuleconst_subgraph_modulehas_folding_been_runrr)selfr r r rr __class__T/var/www/vscode/kcb/lib/python3.10/site-packages/torch/fx/experimental/const_fold.pyrs  zFoldedGraphModule.__init__cs|js|tj|SN)r run_foldingr__call__)rargskwargsrrrr-s zFoldedGraphModule.__call__cszjdus jdur dSjrJd_}fddt|tr0tjfdd|Dn|}tj|dS)NTcsNtjjt|ts|n t|gjj dt|tjr#|j dSddS)N)deviceF) requires_grad) rnn Parameter isinstanceintdetachcloneTensortorr")i)rrr _create_paramCsz4FoldedGraphModule.run_folding.._create_paramcsg|]}|qSrr).0r+)r,rr Lsz1FoldedGraphModule.run_folding..) rrrr%tuplerr# ParameterListsetattr)r folded_attrsparamsr)r,rrr2s     zFoldedGraphModule.run_folding)NNr )__name__ __module__ __qualname____doc__rr#ModulerGraphrstrrrr __classcell__rrrrrs$   rgminline_mod_namec s8t||}t|tjjsJd}|jjD]}|jdkr'|j |kr'|}nq|dus.J|j }|j }id}fdd}|jjD]R} | jdkr^| j |vrS|| j n||| <|d7}qB| jdkrs| j d} t | |} || qB|j||j| |} Wdn1swY| | <qB|jdS) z Given `gm` and some graph module which is called with target name `inline_mod_name`, this helper will inline all of the nodes from that called graph module into `gm`. N call_modulercs|}|j|_|Sr)metacopy)nodenew_nodereplacement_mappingrrreplacement_fnjs z&_inline_module..replacement_fn placeholderoutput)dict named_modulesr%rrrr nodesoptargetrr namerreplace_all_uses_withinserting_before node_copyeliminate_dead_code) r<r= inline_modcall_mod_node_to_replacerA call_mod_argscall_mod_kwargsph_countrE inline_nodeoutputsoutput_replacementsrBrrCr_inline_moduleSs@            r[ mod_tracedrNreturncCstdd|}|drd|}t||r>td|}|dur&|d}n|dd\}}|dt|d}t||s|S) zP Make sure the name is unique (in a module) and can represents an attr. z[^0-9a-zA-Z_]+_rz (.*)_(\d+)$N_1rG)resubisdigithasattrmatchgroupr&)r\rNrebasenumrrrrs      rcpumoduleskip_folding_node_fnrc s2t|tjjstj|}n|}td}|jjD].jdvr!qjdkr/tj  s/q|r6|r6q r;q jdkrGd}q|sPt ||jSdtjjffdd }t|||}d\}}|jt||d } } | ru| jjngD]jd krt|jt| jqw| jjD]jd krt|jt| jqd } |jjD]jd krj|krj} nq| d usJtj|| j} | jjD]Wjd krtjd t} qˈjd krqtfdd| D}|jdksJ| j| j|j}Wd n 1s wYj|_|| jqdtvs+Jt|d}t||| r;tj ntj!|jjD]9jd kr}j|kr}jj|}Wd n 1skwYj|_|nqEt"||rt#|||j$t ||j| j||S)aJ Looks through `module` for any nodes that have all constant attribute inputs and separates them out into their own constant subgraph, and returns a FoldedGraphModule which runs that constant subgraph on the first run to set attributes on the module prior to running the non-constant portion of the graph. F>rHrFget_attrTrAcs|vrdSdS)NrrGrrA) const_nodesrr mod_partitionsz,split_const_subgraphs..mod_partition)submod_0submod_1Nr>rHrrFc3s |] }|jjkr|VqdSr)rNrM)r-nrmrr sz(split_const_subgraphs..multiple_outputs_FX_CONST_FOLDED_ATTRS)%r%rrrsymbolic_tracesetr rKrLall_input_nodesissubset is_impureaddrNoderrpgetattrr1rMrr/nextrPrlr?r@rO erase_nodelocalsrr#r0r$rdr[rR)rjrkrr\found_const_foldingrosplitconst_mod_namenon_const_mod_nameconst_gm non_const_gmcall_const_gm_args root_const_gmrtin_noderBrr2r)rnrArr s                          r )Nri)ratypingrrrtorch.fxr torch.fx.nodertorch.fx.passes.split_moduler__all__rrrr:r[rr#r8r|boolr rrrrs(  B4