o 0 hb-@sddlmZddlZddlZddlZddlmZmZddlZddl m Z er0ddl Z ddl mZeeZ d d!ddZddZ d"d#ddZdS)$) annotationsN)IO TYPE_CHECKING) _type_utils)FileLiketensor torch.Tensornamestrlocationbasepathdtype_overrideonnx.TypeProto | Nonereturnonnx.TensorProtocCsNddl}|durtj|jjntj|j}|dur)||jkr)||}| }||_ | |_ |j |j|j j|_|d|d}|D]\} } |j} | | _t| | _qOtj||} tj| rst| tj| } tj| st | t!| d}|"|j#dd$Wd|S1swY|S)aCreate a TensorProto with external data from a PyTorch tensor. The external data is saved to os.path.join(basepath, location). Args: tensor: Tensor to be saved. name: Name of the tensor (i.e., initializer name in ONNX graph). location: Relative location of the external data file (e.g., "/tmp/initializers/weight_0" when model is "/tmp/model_name.onnx"). basepath: Base path of the external data file (e.g., "/tmp/external_data" while model must be in "/tmp"). Reference for ONNX's external data format: How to load? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L187 How to save? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L43 How to set ONNX fields? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L88 rN)r offsetlengthxbT)force)%onnxjit_type_utils JitScalarTypefrom_onnx_type tensor_type elem_type from_dtypedtypeto TensorProtor onnx_type data_typedimsextendshapeEXTERNAL data_locationuntyped_storagenbytesitems external_dataaddkeyr valueospathjoinexistsremovedirnamemakedirsopenwritenumpytobytes)rr r r r r scalar_type tensor_protokey_value_pairskventryexternal_data_file_pathexternal_data_dir_path data_filerAY/var/www/vscode/kcb/lib/python3.10/site-packages/torch/onnx/_internal/fx/serialization.py'_create_tensor_proto_with_external_datasD              rCcCsdddlm}i}||ddd}|D] }||||<qWd|S1s+wY|S)Nr) safe_openptcpu) frameworkdevice) safetensorsrDkeys get_tensorrF)safetensors_filerDtensorsfr;rArArB$_convert_safetensors_to_torch_formatis   rOFmodel_locationinitializer_locationtorch_state_dictstuple[dict | FileLike, ...] onnx_modelonnx.ModelProtorename_initializerboolNonec Cs ddl}i}ddt|jjD}dd|jjD} |D]} t| tr&| } nft| ttj fr;t |  dr;t | } nQz t j| dd d } WnFttfy} z8d t| vset| tjtfr| r| rtd t| tjtfrw| dt j| dd } n| WYd} ~ nd} ~ ww| D][\} }|r| dd} | | vr| | n| D]}| | s| |r|} | |nqtj|| }dd|jjD}| |vr| ||| <t|| |||| d}|jj |qqtt!|d d}|"D]}|jj|=q|#|tj||dS)aLoad PyTorch tensors from files and add to "onnx_model" as external initializers. Output files: ONNX model file path: ONNX initializer folder: os.path.join(basepath, initializer_location) After running this function, you can do ort_sess = onnxruntime.InferenceSession(os.path.join(basepath, model_location)) to execute the model. Arguments: basepath: Base path of the ONNX external data file (e.g., "/path/to/large_model/"). model_location: Relative location of the ONNX model file. E.g., "model.onnx" so that the model file is saved to "/model.onnx". initializer_location: Relative location of the ONNX initializer folder. E.g., "initializers" so that the initializers are saved to "/initializers/". Note: When initializers are >2GB, must be the same as `model_location`. torch_state_dicts: Dictionaries or files which contain PyTorch tensors to be saved as ONNX initializers. For non-dict arguments, `torch.load` will be used to load them from file-like objects. onnx_model: ONNX model to be saved with external initializers. If an input name matches a tensor loaded from "torch_state_dicts", the tensor will be saved as that input's external initializer. rename_initializer: Replaces "." by "_" for all ONNX initializer names. Not needed by the official torch.onnx.dynamo_export. This is a hack for supporting `FXSymbolicTracer` tracer with fake tensor mode. In short, `FXSymbolicTracer` lifts FX parameters (self.linear_weight) as inputs (`def forward(self, linear_weight)`) and therefore, `.` cannot be used. rNcSsi|]\}}|j|qSrAr ).0idxr;rArArB s z1save_model_with_external_data..cSsh|]}|jqSrArY)rZinputrArArB sz0save_model_with_external_data..z .safetensorsrFT) map_locationmmapz+mmap can only be used with files saved withzFailed to load the checkpoint with memory-map enabled, retrying without memory-map.Consider updating the checkpoint with mmap by using torch.save() on PyTorch version >= 1.6.)r_._cSsi|]}|j|jqSrA)r type)rZr;rArArBr\s)reverse)$r enumerategraph initializerr] isinstancedictr r-PathLikefspathendswithrOtorchload RuntimeError ValueErrorioIOBaserreadableseekablelogwarningseekr(replacer1r.r/rCpopappendsortedrJsave)r rPrQrRrTrVrinitializers_to_be_deletedexisting_initializersonnx_input_namesel state_dicter ronnx_input_namerelative_tensor_file_pathmodel_input_typesr9r[rArArBsave_model_with_external_datavs~'           0  r)N) rrr r r r r r r rrr)F)r r rPr rQr rRrSrTrUrVrWrrX) __future__rrqloggingr-typingrrrm torch.onnxrrr torch.typesr getLogger__name__rurCrOrrArArArBs     T