PgbdZddlZddlmZmZmZmZmZmZddl Z ddl m Z ddl m Z m Z mZddlmZmZmZddlmZmZmZdd lmZmZdd lmZmZdd lmZmZm Z m!Z!m"Z"m#Z#m$Z$dd l%m%Z%m&Z&dd lm'Z'ddl(m)Z)gdZ*ee jVZ,eeeee jVfZ-dee jVej\fde jVde/dee-e-ffdZ0deee jVeej\fdee,de/dee,dffdZ1deede2dee,dffdZ3 d0dedeedee/de/deed eeddfd!Z4 d1ded ed"eedee/de/d#e/d$ee/de/d%e/dee jVdffd&Z5d'Z6d(Z(e)e(_)e jnjqse9d)dd*l:m;Z;mZ>m?Z?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZLmMZMmNZNmOZOmPZPmQZQdd+lRmSZSmTZTmUZUdd,lmVZVd-ZWe jnjZYe jnjeYd.e jnjZ\e jnje\d/y)2a ``torch.autograd`` provides classes and functions implementing automatic differentiation of arbitrary scalar valued functions. It requires minimal changes to the existing code - you only need to declare :class:`Tensor` s for which gradients should be computed with the ``requires_grad=True`` keyword. As of now, we only support autograd for floating point :class:`Tensor` types ( half, float, double and bfloat16) and complex :class:`Tensor` types (cfloat, cdouble). N)castListOptionalSequenceTupleUnion)_vmap_internals)handle_torch_functionhas_torch_functionis_tensor_like)_size_TensorOrTensors_TensorOrTensorsOrGradEdge) forward_ad functionalgraph)detect_anomalyset_detect_anomaly)FunctionNestedIOFunction)_force_original_view_tracking _unsafe_preserve_version_counter enable_gradinference_modeno_gradset_grad_enabledset_multithreading_enabled) gradcheck gradgradcheck_engine_run_backward)Variable)r#rbackward grad_moderrrgradrr rrrrrvariableoutputr&is_grads_batchedreturncddlm}t|tjr[|r t d|j j|j}tj|j|jfS|jr=t||s1|r t d|j}|j}||fS|j}|s |jn|jdd}||fS)Nr NestedTensorz1Batched grads are not supported with GradientEdgez3Batched grads are not supported with Nested Tensor.r)$torch.nested._internal.nested_tensorr- isinstancer GradientEdge RuntimeErrornode_input_metadata output_nrtorchSizeshape is_nested_nested_tensor_size) r(r&r)r- out_metadata out_shape grad_shape reg_out_shapereg_grad_shapes ^/var/www/html/suriana-translation/venv/lib/python3.12/site-packages/torch/autograd/__init__.py_calculate_shaper@;sB&%,,- RS S{{2263C3CD zz,,,-tzz99  6< @ TU U..0 --/ *$$LLM'7TZZTZZ^N . ((outputsgrads.c  g}t||D]\}}tttjt j f|}d}d}t|t j r|jj|j}tj|j}|j} |j}|j} |j r t#dd} nYddlm} t|tjsJ|j} |j(} | xr t||  } | s |j}t|tjrddlm} m}|s|n|d}| r3t|tjsJtj0||}n!|J| |||j3}|s tttjt j f|}t5|||\}}|ret#dt7|j9|zdzt7|zdzt7|j9|zdzt7|zd zt#d t7|j9|zdzt7|zdzt7|j9|zdzt7|zd z| j:|jj:k7rot#d t7|j9|zd zt7|jzdzt7|j9|zd zt7| zd z|j=|f|`t|t j s |j>r&t|t j r|JtAd|D}n/t|tjsJ|jCdk(}|s t#d| jDsd| }t#|t|t j r1|J|J|j=tjF|| |dt|tjsJ|j=tjH|tjJ|j=dtMdtO|jPztS|S)Nz4C++ NestedTensor are not supported with GradientEdgeFrr,) expect_truesym_eqzIf `is_grads_batched=True`, we interpret the first dimension of each grad_output as the batch dimension. The sizes of the remaining dimensions are expected to match the shape of corresponding output, but a mismatch was detected: grad_output[z] has a shape of z and output[z_. If you only want some tensors in `grad_output` to be considered batched, consider using vmap.zMismatch in shape: grad_output[.zuFor complex Tensors, both grad_output and output are required to have the same dtype. Mismatch in dtype: grad_output[z] has a dtype of c3&K|] }|dk( yw)rN).0os r? z_make_grads..s(BAa(Bsrz6grad can be implicitly created only for scalar outputszDgrad can be implicitly created only for real scalar outputs but got )dtypedevice) memory_formatz1gradients can be either Tensors or None, but got )*ziprrr5Tensorrr0r/r2r3r4r6r7rMrNis_nested_tensoris_cpp_nested_tensorr1r.r-r8%torch.fx.experimental.symbolic_shapesrErF is_same_sizesizer@strindex is_complexappend requires_gradallnumelis_floating_pointones ones_likepreserve_format TypeErrortype__name__tuple)rBrCr) new_gradsoutr&out_size out_devicer: out_dtype out_is_nestedout_is_cpp_nestedr-rErF first_grad shape_matchesr;r<out_numel_is_1msgs r? _make_gradsrqXs (*I%(F T5u'9'99:C@ c5-- .8833CMMBLzz,"4"45H$**I%,,J(99M00"J!&  Jc5<<0 00 IMMM - Sjl6S2S $99 dELL ) Q%547J !!#u||444 % 2 23 C  +++ +F8Z__=N,O P  5u/A/A!ABCH(8%5)% :$&5 ekk$/0 1 . .j/*))gmmC01 2. .i. )88$'9ekk$/01-.j/*) ) gmmC01 2 . .i.)  ##tzz'<'<<"7%++d+,-* * $**o & % %'--,-.**)n %      T " \#u112c6G6Gc5#5#56#///%((B(B%BN%c5<<888%(YY[A%5N%&P!22$$-;0's++c5#5#56#///%111$$ $"+#-&c5<<888$$5;P;PQ  &Ct*%%& GFN  rAtensorslengthc`|d|zSt|tjr|fSt|S)NN)r/r5rQre)rrrss r?_tensor_or_tensors_to_tuplervs4'5<<(z >rA grad_tensors retain_graph create_graphgrad_variablesinputsc "tjjr td|,t j dt d||}n td|t|dk(r tdt|tjr|fn t|}t|tjtjfr|fn| t|nd }t|t|}t||d }||}t|||||d d y)a Compute the sum of gradients of given tensors with respect to graph leaves. The graph is differentiated using the chain rule. If any of ``tensors`` are non-scalar (i.e. their data has more than one element) and require gradient, then the Jacobian-vector product would be computed, in this case the function additionally requires specifying ``grad_tensors``. It should be a sequence of matching length, that contains the "vector" in the Jacobian-vector product, usually the gradient of the differentiated function w.r.t. corresponding tensors (``None`` is an acceptable value for all tensors that don't need gradient tensors). This function accumulates gradients in the leaves - you might need to zero ``.grad`` attributes or set them to ``None`` before calling it. See :ref:`Default gradient layouts` for details on the memory layout of accumulated gradients. .. note:: Using this method with ``create_graph=True`` will create a reference cycle between the parameter and its gradient which can cause a memory leak. We recommend using ``autograd.grad`` when creating the graph to avoid this. If you have to use this function, make sure to reset the ``.grad`` fields of your parameters to ``None`` after use to break the cycle and avoid the leak. .. note:: If you run any forward ops, create ``grad_tensors``, and/or call ``backward`` in a user-specified CUDA stream context, see :ref:`Stream semantics of backward passes`. .. note:: When ``inputs`` are provided and a given input is not a leaf, the current implementation will call its grad_fn (even though it is not strictly needed to get this gradients). It is an implementation detail on which the user should not rely. See https://github.com/pytorch/pytorch/pull/60521#issuecomment-867061780 for more details. Args: tensors (Sequence[Tensor] or Tensor): Tensors of which the derivative will be computed. grad_tensors (Sequence[Tensor or None] or Tensor, optional): The "vector" in the Jacobian-vector product, usually gradients w.r.t. each element of corresponding tensors. None values can be specified for scalar Tensors or ones that don't require grad. If a None value would be acceptable for all grad_tensors, then this argument is optional. retain_graph (bool, optional): If ``False``, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to ``True`` is not needed and often can be worked around in a much more efficient way. Defaults to the value of ``create_graph``. create_graph (bool, optional): If ``True``, graph of the derivative will be constructed, allowing to compute higher order derivative products. Defaults to ``False``. inputs (Sequence[Tensor] or Tensor or Sequence[GradientEdge], optional): Inputs w.r.t. which the gradient be will accumulated into ``.grad``. All other Tensors will be ignored. If not provided, the gradient is accumulated into all the leaf Tensors that were used to compute the :attr:`tensors`. zbackward() called inside a functorch transform. This is not supported, please use functorch.grad or functorch.vjp instead or call backward() outside of functorch transforms.Nz;`grad_variables` is deprecated. Use `grad_tensors` instead. stacklevelzw`grad_tensors` and `grad_variables` (deprecated) arguments both passed to `backward()`. Please only use `grad_tensors`.rz2`inputs` argument to `backward()` cannot be empty.rIFr)T)allow_unreachableaccumulate_grad)r5_C _are_functorch_transforms_activer1warningswarn FutureWarninglenr/rQrerr0rvrqr")rrrwrxryrzr{ grad_tensors_s r?r$r$s"@ xx002 B  ! I   )L&  c&kQ.OPP&w =wj5>G fu||U-?-?@ A    6]   0 c'lKMOM# rA grad_outputs only_inputs allow_unusedmaterialize_gradsc |rdur td|tsttjrAt t ttjttjffn ttsttjrt tfn ttdD} tdD} | | z} t| rtt| |||| S|stj dt"dt%|t'} t)| | } |r)fd } t+j,| d d d | nt/| d|rNt1fdt3t'Dr t5dtdt7DS)a Compute and return the sum of gradients of outputs with respect to the inputs. ``grad_outputs`` should be a sequence of length matching ``output`` containing the "vector" in vector-Jacobian product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn't require_grad, then the gradient can be ``None``). .. note:: If you run any forward ops, create ``grad_outputs``, and/or call ``grad`` in a user-specified CUDA stream context, see :ref:`Stream semantics of backward passes`. .. note:: ``only_inputs`` argument is deprecated and is ignored now (defaults to ``True``). To accumulate gradient for other parts of the graph, please use ``torch.autograd.backward``. Args: outputs (sequence of Tensor or GradientEdge): outputs of the differentiated function. inputs (sequence of Tensor or GradientEdge): Inputs w.r.t. which the gradient will be returned (and not accumulated into ``.grad``). grad_outputs (sequence of Tensor): The "vector" in the vector-Jacobian product. Usually gradients w.r.t. each output. None values can be specified for scalar Tensors or ones that don't require grad. If a None value would be acceptable for all grad_tensors, then this argument is optional. Default: None. retain_graph (bool, optional): If ``False``, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to ``True`` is not needed and often can be worked around in a much more efficient way. Defaults to the value of ``create_graph``. create_graph (bool, optional): If ``True``, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: ``False``. allow_unused (Optional[bool], optional): If ``False``, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to the value of ``materialize_grads``. is_grads_batched (bool, optional): If ``True``, the first dimension of each tensor in ``grad_outputs`` will be interpreted as the batch dimension. Instead of computing a single vector-Jacobian product, we compute a batch of vector-Jacobian products for each "vector" in the batch. We use the vmap prototype feature as the backend to vectorize calls to the autograd engine so that this computation can be performed in a single call. This should lead to performance improvements when compared to manually looping and performing backward multiple times. Note that due to this feature being experimental, there may be performance cliffs. Please use ``torch._C._debug_only_display_vmap_fallback_warnings(True)`` to show any performance warnings and file an issue on github if warnings exist for your use case. Defaults to ``False``. materialize_grads (bool, optional): If ``True``, set the gradient for unused inputs to zero instead of None. This is useful when computing higher-order derivatives. If ``materialize_grads`` is ``True`` and ``allow_unused`` is ``False``, an error will be raised. Defaults to ``False``. FzhExpected allow_unused to be True or not passed when materialize_grads=True, but got: allow_unused=False.c38K|]}t|s|ywrur rJis r?rLzgrad..s>AN1,=a>c38K|]}t|s|ywrurrs r?rLzgrad..s<1.*;Q.vjps%' % rArT)allow_none_pass_throughrc3NK|]}|duxrt| ywrur)rJrr{results r?rLzgrad..s6  1I  ?nVAY&?"? ? s"%zGmaterialize_grads cannot be used when the given input is a GradientEdgec3XK|]"\}}||ntj|d$yw)NT)r[)r5 zeros_like)rJr(inputs r?rLzgrad..s;  ! !!%t< = s(*) ValueErrorr r/rr0rrrr5rQrerr r r&rrrrvrrqr _vmapr"anyranger1rP)rBr{rrxryrrr)r t_outputst_inputsoverridable_args grad_outputs_rrs`` `` ` @r?r&r&fsD\U2 +  ( g*We6H6H"I (5<<((53E3E*FF G' .fFE4F4F!G06)<v>>>I<<tjjSru)r#_execution_engineis_checkpoint_validrIrAr?_is_checkpoint_validrs  % % 9 9 ;;rActd)NzItorch.autograd.variable(...) is deprecated, use torch.tensor(...) instead)r1)argskwargss r?r'r's S rAzautograd initialization failed)_add_metadata_json_disable_profiler_disable_profiler_legacy_enable_profiler_enable_profiler_legacy_enable_record_function_get_sequence_nr _kineto_step _KinetoEvent _pop_saved_tensors_default_hooks_prepare_profiler_profiler_enabled_ProfilerResult!_push_saved_tensors_default_hooks _record_function_with_args_enter_record_function_with_args_exit_set_empty_test_observer_supported_activities_toggle_collection_dynamic DeviceTypekineto_available ProfilerEvent SavedTensor)ProfilerActivityProfilerConfig ProfilerState)profilerczt|ts tdtjj ||y)Nzcls isn't a typeinfo object)r/rcr1r5r_register_py_class_for_device)rNclss r?$_register_py_tensor_class_for_devicerIs, c4 899 HH**637rAz4Returns True if multithreading is currently enabled.z1Returns True if view-replay is currently enabled.)NNFNN)NNFTNFF)]__doc__rtypingrrrrrrr5r torch.overridesr r r torch.typesr rrrrr anomaly_moderrfunctionrrr%rrrrrrrrr r"r'r#__all__rQ_OptionalTensor_ShapeorNestedShaper0boolr@rqintrvr$r&rr_autograd_initr1torch._C._autogradrrrrrrrrrrrrrrrrrrrrrrrtorch._C._profilerrrrrr_is_multithreading_enabledis_multithreading_enabled _add_docstr_is_view_replay_enabledis_view_replay_enabledrIrAr?rs'?? !UUKK++<00' &5<<(E8E?ELL@A) %,, 2 22 3) ,,))  3 34 ):M 8ELL)8E4F4F+GG HM O $MM ?C  M` & '14 ?C  04#'1537 q q+,q4.q q -. q / 0 q qn04#'#'"#a 'a &a+,a4. a  a  a4.aaa 5<< ab< xx 7 882ON8 "HH??U99OrA