# Defines derivative formulas and Python signatures of methods on Variable

#

# Note about possibly confusing nomenclature: An 'output gradient' is the

# gradient of an output of a forward function. Output gradients are used as

# the inputs to backward functions. `grads` is a vector of output gradients,

# and `grad == grads[0]`, in all the derivative formulas in this file.

# An 'input gradient' is the gradient of an input to a forward function.

# Input gradients are the outputs of backward functions, corresponding to the

# input names included in the derivative formulas defined in this file.

# Also, every time we talk computing "gradient" we actually mean computing

# the vector jacobian product using the given 'output gradient' as the vector.

#

# Each entry consists of:

# - A 'name', which specifies the ATen name of the function you

# are defining derivatives for, and an argument specification.

# - One or more gradients entries, mapping differentiable input

# names to a formula specifying how to compute its gradient.

# Note that a single gradient entry can specify the gradient

# formula for multiple input names, by specifying a key

# "input1, input2" (see atan2 for an example).

# - An argument can be flagged as 'non_differentiable'.

# - Optional entry with key 'output_differentiability' and value a list of the

# same length as the number of outputs from the forward function. The list

# should contain only booleans, specifying whether each of the output Tensor

# is differentiable.

# If it is not specified for a function that returns multiple elements but

# uses `grad` instead of `grads[idx]`, then all but the first output will

# be marked as non-differentiable.

# If None of the output is differentiable, you can also add the function

# name to `gen_variable_type.py`'s `DONT_REQUIRE_DERIVATIVE` list.

#

# There are two cases for Tensor and TensorList arguments here:

# - If that argument is differentiable, in the sense that a gradient with respect

# to that argument could exist. You should either:

# - Specify the formula for that gradient

# - Specify not_implemented("function_name") as a formula to say that this is not

# implement yet (but might be in the future and the user can request that on an issue)

# - If that argument is not differentiable, because it is not a floating point dtype or the

# function is not differentiable with respect to that argument for

# example. You should either:

# - Do not specify any formula for this argument

# - Specify explicitly that this argument is "non_differentiable". Note that in this case,

# we trust you that this argument will never have requires_grad=True and it will be silently

# ignored if it does.

#

# If a function has out-of-place and in-place variants, then the derivative

# definition for the in-place variant is optional. It will default to the

# definition for the out-of-place variant. Note that _out variants are never

# differentiable.

#

# Gradient expressions are standard C++ expressions operating on ATen

# variables. In a gradient expression, the following variables are in

# scope:

#

# - 'grad', the gradient of the output (often spelled grad_output

# in Python) which we are going to left-multiply.

#

# When a function returns multiple *differentiable* outputs,

# you can refer to the gradients of each outputs using 'grads',

# e.g., 'grads[0]', 'grads[1]'.

#

# When a function returns *one* differentiable output (the

# first output) and some more nondifferentiable outputs,

# you MUST refer to the gradient of the differentiable output with

# 'grad' (this case is special-cased in our code generation).

#

# Note that the number of differentibale outputs can be modified by the

# 'output_differentiability' entry (see above).

#

# - Any of the input arguments, tensor or non-tensor, including

# argument names that only appear in Declarations.yaml, e.g. 'output'.

#

# - 'result', representing the result of evaluating the forward

# expression for ATen native function declarations. If the forward

# expression outputs a tuple, use 'resultX' instead to access the

# X-th entry

#

# - 'grad_input_mask', a std::array<bool, n>, specifies which input

# gradients are actually needed. For example, in the entry

# `input0, input1: foo(grad_input_mask)`, `grad_input_mask` is a size

# two array, where `grad_input_mask[0]` is true if `input0` requires

# grad, and `grad_input_mask[1]` is true if `input1` requires grad.

#

# (NB: if your function computes gradient for a list of tensors,

# the `grad_input_mask` will only have a single entry for the list

# specifying if either zero or at least one tensor from the list requires

# grad. If we want to support more fine-grained signalling,

# we'll need some alternate variable which is not a std::array)

#

# - 'retain_variables', a bool which is true if a user has specified

# that saved variables should be retained in case the backwards is

# run again later. This allows an optimization where we can

# destroy saved buffers if we know variables are not going to be retained,

# e.g., it is used by _cudnn_rnn

#

# If you need a complex expression, e.g., with local variables,

# write a _backward function in torch/csrc/autograd/FunctionsManual.cpp

# and invoke it from here. By the way, go read

# https://github.com/zdevito/ATen/issues/163; this describes an

# important hazard that occurs when porting backwards from Python to C++

#

# Double backwards gradient expressions can be somewhat confusing;

# the most important thing to remember is: (1) you need to define a

# derivative formula for every input, including inputs named things

# like 'grad_output', and (2) the gradient to multiply with is always

# called 'grad' (even though it really is a grad-grad).

#

# You can also add forward derivative definition by defining a formula for

# a returned value (in general "result" if the name is not specified). This

# formula works the same way as the backward one and advanced implementations

# should also be placed in the FunctionsManual file.

# This formula should compute a single Jacobian vector product using the (primal)

# value of the argument "foo_p", its forward grad "foo_t" and the result of the

# function as "result".

# Note that the forward derivative can be automatically generated in two cases:

# - if your function is linear (NOT affine or multi-linear), then you can

# specify so by just using the string "auto_linear" for the formula.

# - if your function is applied element wise (and has a single input), you

# can specify so by just using the string "auto_element_wise" for the formula.

#

# Note that to avoid unpacking overhead, functions taking TensorList as inputs

# will always have their forward grad formula called. This function is responsible

# to check if any computation is needed and should return an undefined Tensor when

# there is nothing to do. You can check "cat_forward" for a full example.

#

# NB: There are a number of gradient definitions in here which are bogus

# (implemented using zeros_like). These gradients are (hopefully) not

# used by our frontend. You MUST check the frontend code; search for

# OpName.apply to see if it's still using a legacy Python style API.

#

# Note: Returning views.

# The following cases exist:

# - If a function returns no view, it can have arbitrary outputs.

# - If a function return at least one Tensor that is a differentiable view

# of one of its input:

# - If there is only one differentiable output, this Tensor is marked as a

# differentiable view. (alias or transpose for example)

# - If there are more than one differentiable output, by default all the views are

# marked as differentiable views and created with allow_rebase_history=false.

# Meaning that any inplace operation on it will raise an error. (unbind for example)

#

# Notes about undefined output gradients:

# All backward functions must support all combinations of undefined output

# gradient Tensors, where `grad[i].defined() == false`. Depending on the

# number of input and output grads your derivative formula uses, code

# generation may automatically add some level of undefined grad support,

# according to these three cases:

#

# * 1 input grad and 1 output grad:

# Complete undefined grad support is automatically added, so you

# shouldn't have to think about it, unless there is a bug in the code

# generation.

#

# * 1 input grad and multiple output grads:

# Undefined grad support is automatically added ONLY in the case where

# all output grads are undefined. You will have to add explicit support

# for cases where a subset of output grads is undefined.

#

# * multiple input grads:

# No automatic support, so you will need to add it.

#

# If your derivative formula uses more than one output grad, it is usually

# preferable to add undefined grad support in the backward function itself

# (if you're using one), rather than in the derivative formula in this file.

#

# Undefined Tensors are created with the default constructor `at::Tensor()`.

# It is an efficient way to represent a Tensor filled with zeros because

# the Tensor holds no sizing information and no Storage data is allocated.

# But consequentially, Tensor operations cannot be performed on them.

# Therefore, your backward function should treat an undefined output grad as

# a zero, and it needs to be a special case.

#

# If all output grads are undefined, then it should be correct for the

# backward function to return undefined input grads. Since we use the chain

# rule, output grads equal to zero should result in input grads equal to zero,

# unless there is some rare special case.

#

# If a subset of output grads is undefined, then it may be acceptable for

# the backward function to return undefined input grads--it depends on the

# specific function, so you'll have to determine that yourself. If returning

# an undefined Tensor is correct for a given input grad, it is also logically

# correct to return a defined grad full of zeros, but that would not be

# preferable since it would be less efficient.

#

# NB: The parameter names here MUST be consistent with the parameter names

# in Decalarations.yaml

- name: abs(Tensor self) -> Tensor

self: grad * self.sgn()

result: handle_r_to_c(result.scalar_type(), self_t.conj() * self_p.sgn())

- name: acos(Tensor self) -> Tensor

self: grad * -((-self * self + 1).rsqrt()).conj()

result: auto_element_wise

- name: add.Tensor(Tensor self, Tensor other, *, Scalar alpha=1) -> Tensor

self: handle_r_to_c(self.scalar_type(), grad)

other: handle_r_to_c(other.scalar_type(), maybe_multiply(grad, alpha.conj()))

result: self_t + maybe_multiply(other_t, alpha)

- name: add.Scalar(Tensor self, Scalar other, Scalar alpha=1) -> Tensor

self: handle_r_to_c(self.scalar_type(), grad)

result: self_t

- name: addbmm(Tensor self, Tensor batch1, Tensor batch2, *, Scalar beta=1, Scalar alpha=1) -> Tensor

self: maybe_multiply(grad, beta.conj())

batch1: grad.unsqueeze(0).expand({ batch1.size(0), batch1.size(1), batch2.size(2) }).bmm(batch2.transpose(1, 2).conj()) * alpha.conj()

batch2: batch1.transpose(1, 2).conj().bmm(grad.unsqueeze(0).expand({ batch1.size(0), batch1.size(1), batch2.size(2) })) * alpha.conj()

result: maybe_multiply(self_t, beta) + maybe_multiply(batch1_t.bmm(batch2_p).sum(0), alpha) + maybe_multiply(batch1_p.bmm(batch2_t).sum(0), alpha)

- name: addcdiv(Tensor self, Tensor tensor1, Tensor tensor2, *, Scalar value=1) -> Tensor

self: handle_r_to_c(self.scalar_type(), grad)

tensor1: handle_r_to_c(tensor1.scalar_type(), grad * (value / tensor2).conj())

tensor2: handle_r_to_c(tensor2.scalar_type(), -grad * (value * tensor1 / (tensor2 * tensor2)).conj())

result: self_t + maybe_multiply(tensor1_t / tensor2_p, value) - maybe_multiply(tensor2_t * (tensor1_p / tensor2_p) / tensor2_p, value)

- name: addcmul(Tensor self, Tensor tensor1, Tensor tensor2, *, Scalar value=1) -> Tensor

self: handle_r_to_c(self.scalar_type(), grad)

tensor1: handle_r_to_c(tensor1.scalar_type(), grad * (tensor2 * value).conj())

tensor2: handle_r_to_c(tensor2.scalar_type(), grad * (tensor1 * value).conj())

result: self_t + maybe_multiply(tensor1_t * tensor2_p, value) + maybe_multiply(tensor2_t * tensor1_p, value)

- name: addmm(Tensor self, Tensor mat1, Tensor mat2, *, Scalar beta=1, Scalar alpha=1) -> Tensor

self: maybe_multiply(grad, beta.conj())

mat1: mm_mat1_backward(grad, mat2, mat1.sizes(), mat1.strides(), alpha)

mat2: mm_mat2_backward(grad, mat1, mat2.sizes(), mat2.strides(), alpha)

result: maybe_multiply(self_t, beta) + maybe_multiply(mat1_t.mm(mat2_p), alpha) + maybe_multiply(mat1_p.mm(mat2_t), alpha)

- name: _sparse_addmm(Tensor self, Tensor sparse, Tensor dense, *, Scalar beta=1, Scalar alpha=1) -> Tensor

self: maybe_multiply(grad, beta)

sparse: _sparse_addmm_sparse_backward(grad, sparse, dense, alpha)

dense: mm_mat2_backward(grad, sparse, dense.sizes(), dense.strides(), alpha)

- name: addmv(Tensor self, Tensor mat, Tensor vec, *, Scalar beta=1, Scalar alpha=1) -> Tensor

self: maybe_multiply(grad, beta.conj())

mat: grad.ger(vec.conj()) * alpha.conj()

vec: mat.t().conj().mv(grad) * alpha.conj()

result: maybe_multiply(self_t, beta) + maybe_multiply(mat_t.mv(vec_p), alpha) + maybe_multiply(mat_p.mv(vec_t), alpha)

- name: addr(Tensor self, Tensor vec1, Tensor vec2, *, Scalar beta=1, Scalar alpha=1) -> Tensor

self: maybe_multiply(grad, beta.conj())

vec1: grad.mv(vec2.conj()) * alpha.conj()

vec2: grad.t().mv(vec1.conj()) * alpha.conj()

result: maybe_multiply(self_t, beta) + maybe_multiply(vec1_t.outer(vec2_p), alpha) + maybe_multiply(vec1_p.outer(vec2_t), alpha)

- name: affine_grid_generator(Tensor theta, int[] size, bool align_corners) -> Tensor

theta: affine_grid_generator_backward(grad, size, align_corners)

- name: alias(Tensor(a) self) -> Tensor(a)

self: grad

result: self_t

- name: angle(Tensor self) -> Tensor

self: angle_backward(grad, self)

result: handle_r_to_c(result.scalar_type(), angle_backward(self_t, self_p))

# The four items below are necessary because TensorIterator doesn't work on

# Variables (codegen does not unwrap the input Tensor for all() and any() ).

- name: any(Tensor self) -> Tensor

self: not_implemented("any")

- name: any.dim(Tensor self, int dim, bool keepdim=False) -> Tensor

self: not_implemented("any")

- name: all(Tensor self) -> Tensor

self: not_implemented("all")

- name: all.dim(Tensor self, int dim, bool keepdim=False) -> Tensor

self: not_implemented("all")

- name: acosh(Tensor self) -> Tensor

self: grad * (self.pow(2) - 1).rsqrt().conj()

result: auto_element_wise

- name: acosh_(Tensor(a!) self) -> Tensor(a!)

self: not_implemented("inplace version of acosh")

- name: asinh(Tensor self) -> Tensor

self: grad * (self.pow(2) + 1).rsqrt().conj()

result: auto_element_wise

- name: asinh_(Tensor(a!) self) -> Tensor(a!)

self: not_implemented("inplace version of asinh")

- name: atanh(Tensor self) -> Tensor

self: grad * 1 / (1 - self.pow(2)).conj()

result: auto_element_wise

- name: atanh_(Tensor(a!) self) -> Tensor(a!)

self: not_implemented("inplace version of atanh")

- name: as_strided(Tensor(a) self, int[] size, int[] stride, int? storage_offset=None) -> Tensor(a)

self: as_strided_backward(grad, TensorGeometry(self), size, stride, storage_offset)

- name: asin(Tensor self) -> Tensor

self: grad * (-self * self + 1).rsqrt().conj()

result: auto_element_wise

- name: atan(Tensor self) -> Tensor

self: grad / (self * self + 1).conj()

result: auto_element_wise

- name: atan2(Tensor self, Tensor other) -> Tensor

self, other: atan2_backward(grad, self, other, grad_input_mask)

- name: baddbmm(Tensor self, Tensor batch1, Tensor batch2, *, Scalar beta=1, Scalar alpha=1) -> Tensor

self: maybe_multiply(grad, beta.conj())

batch1: grad.bmm(batch2.transpose(1, 2).conj()) * alpha.conj()

batch2: batch1.transpose(1, 2).conj().bmm(grad) * alpha.conj()

result: maybe_multiply(self_t, beta) + maybe_multiply(batch1_t.bmm(batch2_p), alpha) + maybe_multiply(batch1_p.bmm(batch2_t), alpha)

- name: bernoulli(Tensor self, *, Generator? generator=None) -> Tensor

self: zeros_like(grad)

- name: bernoulli_.Tensor(Tensor(a!) self, Tensor p, *, Generator? generator=None) -> Tensor(a!)

self: zeros_like(grad)

p: zeros_like(p)

- name: bernoulli_.float(Tensor(a!) self, float p=0.5, *, Generator? generator=None) -> Tensor(a!)

self: zeros_like(grad)

- name: bmm(Tensor self, Tensor mat2) -> Tensor

self: grad.bmm(mat2.transpose(1, 2).conj())

mat2: self.transpose(1, 2).conj().bmm(grad)

result: self_t.bmm(mat2_p) + self_p.bmm(mat2_t)

- name: cat(Tensor[] tensors, int dim=0) -> Tensor

tensors: cat_tensors_backward(grad, to_args_sizes(tensors), to_args_scalartypes(tensors), dim)

result: cat_jvp(tensors, dim)

- name: cauchy_(Tensor(a!) self, float median=0, float sigma=1, *, Generator? generator=None) -> Tensor(a!)

self: zeros_like(grad)

- name: ceil(Tensor self) -> Tensor

self: zeros_like(grad)

result: auto_element_wise

- name: cholesky(Tensor self, bool upper=False) -> Tensor

self: cholesky_backward(grad, upper, result)

- name: linalg_cholesky_ex(Tensor self, *, bool upper=False, bool check_errors=False) -> (Tensor L, Tensor info)

self: cholesky_backward(grad, upper, L)

L: cholesky_jvp(self_t, L, upper)

- name: cholesky_solve(Tensor self, Tensor input2, bool upper=False) -> Tensor

self, input2: cholesky_solve_backward(grad, self, input2, result, upper)

- name: cholesky_inverse(Tensor self, bool upper=False) -> Tensor

self: cholesky_inverse_backward(grad, self, upper, result)

# For clamp, gradient is not defined at the boundaries. But empirically it's helpful

# to be able to get gradient on min and max, so we return the subgradient 1 for these cases.

- name: clamp.Tensor(Tensor self, Tensor? min=None, Tensor? max=None) -> Tensor

self: clamp_backward(grad, self, min, max)

min, max: clamp_backward_min_max(grad, self, min, max, grad_input_mask)

- name: clamp(Tensor self, Scalar? min=None, Scalar? max=None) -> Tensor

self: clamp_backward(grad, self, min, max)

result: auto_element_wise

- name: clamp_min(Tensor self, Scalar min) -> Tensor

self: where(self >= min, grad, at::scalar_tensor(0., grad.options()))

result: auto_element_wise

- name: clamp_min.Tensor(Tensor self, Tensor min) -> Tensor

self: where(self >= min, grad, at::scalar_tensor(0., grad.options()))

min: where(self < min, grad, at::scalar_tensor(0., grad.options()))

result: where(self_p >= min_p, self_t, at::scalar_tensor(0., self_p.options())) + where(self_p < min_p, min_t, at::scalar_tensor(0., self_p.options()))

- name: clamp_max(Tensor self, Scalar max) -> Tensor

self: where(self <= max, grad, at::scalar_tensor(0., grad.options()))

result: auto_element_wise

- name: clamp_max.Tensor(Tensor self, Tensor max) -> Tensor

self: where(self <= max, grad, at::scalar_tensor(0., grad.options()))

max: where(self > max, grad, at::scalar_tensor(0., grad.options()))

result: where(self_p <= max_p, self_t, at::scalar_tensor(0., self_p.options())) + where(self_p > max_p, max_t, at::scalar_tensor(0., self_p.options()))

- name: clone(Tensor self, *, MemoryFormat? memory_format=None) -> Tensor

self: grad

result: auto_linear

- name: _to_copy(Tensor self, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None, bool non_blocking=False, MemoryFormat? memory_format=None) -> Tensor

self: grad.to(self.options(), /*non_blocking*/false, /*copy*/false)

result: _to_copy(self_t, dtype, layout, device, pin_memory, non_blocking, memory_format)

# The condition is: if dtype is not nullopt, then isDifferentiableType(*dtype)

# (If dtype IS nullopt, we rely on the regular check that any input requires grad).

output_differentiability: ["!dtype || isDifferentiableType(*dtype)"]

- name: _coalesce(Tensor self) -> Tensor

self: grad

- name: complex(Tensor real, Tensor imag) -> Tensor

real: at::real(grad)

imag: at::imag(grad)

result: at::complex(real_t, imag_t)

- name: polar(Tensor abs, Tensor angle) -> Tensor

abs, angle: polar_backward(grad, result)

- name: _conj(Tensor(a) self) -> Tensor(a)

self: grad.conj()

result: self_t.conj()

- name: _neg_view(Tensor(a) self) -> Tensor(a)

self: grad.neg()

result: self_t.neg()

- name: _conj_physical(Tensor self) -> Tensor

self: grad.conj_physical()

result: self_t.conj_physical()

- name: conj_physical_(Tensor(a!) self) -> Tensor(a!)

self: grad.conj_physical()

result: self_t.conj_physical_()

- name: copysign.Tensor(Tensor self, Tensor other) -> Tensor

self: copysign_tensor_self_backward(grad, self, result)

other: zeros_like(other)

result: copysign_tensor_self_backward(self_t, self_p, result)

- name: copysign.Scalar(Tensor self, Scalar other) -> Tensor

self: copysign_tensor_self_backward(grad, self, result)

result: auto_element_wise

- name: cos(Tensor self) -> Tensor

self: grad * -self.sin().conj()

result: auto_element_wise

- name: cosh(Tensor self) -> Tensor

self: grad * self.sinh().conj()

result: auto_element_wise

- name: count_nonzero.dim_IntList(Tensor self, int[] dim) -> Tensor

output_differentiability: [False]

- name: count_nonzero(Tensor self, int? dim=None) -> Tensor

output_differentiability: [False]

- name: cross(Tensor self, Tensor other, int? dim=None) -> Tensor

self: other.conj().cross(grad, dim)

other: grad.cross(self.conj(), dim)

result: "at::cross(self_t, other_p, dim) + at::cross(self_p, other_t, dim)"

- name: logcumsumexp(Tensor self, int dim) -> Tensor

self: logcumsumexp_backward(grad, self, result, dim)

- name: cumprod(Tensor self, int dim, *, ScalarType? dtype=None) -> Tensor

self: cumprod_backward(grad.to(self.scalar_type()), self, dim, result)

result: "cumprod_jvp(self_t, self_p, result, dim).to(dtype.has_value() ? *dtype : self_p.scalar_type())"

- name: cumsum(Tensor self, int dim, *, ScalarType? dtype=None) -> Tensor

self: cumsum_backward(grad.to(self.scalar_type()), dim)

result: auto_linear

- name: cummax(Tensor self, int dim) -> (Tensor values, Tensor indices)

self: cummaxmin_backward(grad, self, indices, dim)

values: self_t.gather(dim, indices)

- name: cummin(Tensor self, int dim) -> (Tensor values, Tensor indices)

self: cummaxmin_backward(grad, self, indices, dim)

values: self_t.gather(dim, indices)

- name: conv_tbc(Tensor self, Tensor weight, Tensor bias, int pad=0) -> Tensor

self, weight, bias: "grad.defined() ? conv_tbc_backward(grad, self, weight, bias, pad) : std::tuple<Tensor, Tensor, Tensor>()"

- name: _ctc_loss(Tensor log_probs, Tensor targets, int[] input_lengths, int[] target_lengths, int blank=0, bool zero_infinity=False) -> (Tensor, Tensor)

log_probs: _ctc_loss_backward(grad, log_probs, targets, input_lengths, target_lengths, result0, result1, blank, zero_infinity)

- name: deg2rad(Tensor self) -> Tensor

self: deg2rad_backward(grad)

result: auto_element_wise

- name: _det_lu_based_helper(Tensor self) -> (Tensor det, Tensor lu, Tensor pivs)

self: _det_lu_based_helper_backward(grad, det, self, lu, pivs)

output_differentiability: [True, False, False]

- name: diag(Tensor self, int diagonal=0) -> Tensor

self: diag_backward(grad, self.sizes(), diagonal)

- name: diagonal(Tensor(a) self, int offset=0, int dim1=0, int dim2=1) -> Tensor(a)

self: diagonal_backward(grad, self.sizes(), offset, dim1, dim2)

result: auto_linear

- name: diagonal_backward(Tensor grad_output, int[] input_sizes, int offset, int dim1, int dim2) -> Tensor

grad_output: grad.diagonal(offset, dim1, dim2)

result: auto_linear

- name: dist(Tensor self, Tensor other, Scalar p=2) -> Tensor

self: norm_backward(grad, self - other, p, result)

other: -norm_backward(grad, self - other, p, result)

# The backward formula is done in this order to improve numerical stability

# of the higher order derivatives, see https://github.com/pytorch/pytorch/issues/43414

# Note that we don't use "result" because saving it would be BC-breaking when it is used in an inplace operation later

- name: div.Tensor(Tensor self, Tensor other) -> Tensor

self: div_tensor_self_backward(grad, other, self.scalar_type())

other: div_tensor_other_backward(grad, self, other)

result: self_t / other_p - other_t * (self_p / other_p) / other_p

- name: div.Scalar(Tensor self, Scalar other) -> Tensor

self: div_tensor_self_backward(grad, at::scalar_to_tensor(other), self.scalar_type())

result: self_t / other

- name: div.Tensor_mode(Tensor self, Tensor other, *, str? rounding_mode) -> Tensor

self: div_tensor_self_backward(grad, other, self.scalar_type(), rounding_mode)

other: div_tensor_other_backward(grad, self, other, rounding_mode)

result: "rounding_mode.has_value() ? result.new_zeros(result.sizes()) : self_t / other_p - other_t * (self_p / other_p) / other_p"

- name: div.Scalar_mode(Tensor self, Scalar other, *, str? rounding_mode) -> Tensor

self: div_tensor_self_backward(grad, at::scalar_to_tensor(other), self.scalar_type(), rounding_mode)

result: "rounding_mode.has_value() ? result.new_zeros(result.sizes()) : self_t / other"

- name: dot(Tensor self, Tensor tensor) -> Tensor

self: grad * tensor.conj()

tensor: grad * self.conj()

result: at::dot(self_t, tensor_p) + at::dot(self_p, tensor_t)

- name: vdot(Tensor self, Tensor other) -> Tensor

self: grad.conj() * other

other: grad * self

result: at::vdot(self_t, other_p) + at::vdot(self_p, other_t)

- name: _fused_dropout(Tensor self, float p, Generator? generator=None) -> (Tensor, Tensor)

self: _fused_dropout_backward(grad, result1, p)

- name: eig(Tensor self, bool eigenvectors=False) -> (Tensor eigenvalues, Tensor eigenvectors)

self: eig_backward(grads, self, eigenvectors, eigenvalues, eigenvectors_return)

- name: eq_.Scalar(Tensor(a!) self, Scalar other) -> Tensor(a!)

self: zeros_like(self)

- name: eq_.Tensor(Tensor(a!) self, Tensor other) -> Tensor(a!)

self: zeros_like(self)

other: zeros_like(other)

- name: erf(Tensor self) -> Tensor

self: 2.0 / sqrt(M_PI) * exp(-(self.pow(2))) * grad

- name: erfc(Tensor self) -> Tensor

self: -2.0 / sqrt(M_PI) * exp(-(self.pow(2))) * grad

- name: special_erfcx(Tensor self) -> Tensor

self: (2.0 * self * result - 2.0 / sqrt(M_PI)) * grad

- name: erfinv(Tensor self) -> Tensor

self: 0.5 * sqrt(M_PI) * exp(self.erfinv().pow(2)) * grad

- name: exp(Tensor self) -> Tensor

self: grad * result.conj()

result: auto_element_wise

- name: exp2(Tensor self) -> Tensor

self: grad * result * M_LN2

result: auto_element_wise

- name: expm1(Tensor self) -> Tensor

self: grad * (result + 1)

result: auto_element_wise

- name: expand(Tensor(a) self, int[] size, *, bool implicit=False) -> Tensor(a)

self: at::sum_to(grad, self.sizes())

result: auto_linear

- name: exponential_(Tensor(a!) self, float lambd=1, *, Generator? generator=None) -> Tensor(a!)

self: zeros_like(grad)

- name: fake_quantize_per_tensor_affine_cachemask(Tensor self, float scale, int zero_point, int quant_min, int quant_max) -> (Tensor output, Tensor mask)

self: fake_quantize_per_tensor_affine_cachemask_backward(grad, mask)

- name: _fake_quantize_per_tensor_affine_cachemask_tensor_qparams(Tensor self, Tensor scale, Tensor zero_point, Tensor fake_quant_enabled, int quant_min, int quant_max) -> (Tensor output, Tensor mask)

self: fake_quantize_per_tensor_affine_cachemask_backward(grad, mask)

- name: _fake_quantize_learnable_per_tensor_affine(Tensor self, Tensor scale, Tensor zero_point, int quant_min, int quant_max, float grad_factor=1.0) -> Tensor

self, scale, zero_point: "grad.defined() ? _fake_quantize_learnable_per_tensor_affine_backward(grad, self, scale, zero_point, quant_min, quant_max, grad_factor) : std::tuple<Tensor, Tensor, Tensor>()"

- name: fake_quantize_per_channel_affine_cachemask(Tensor self, Tensor scale, Tensor zero_point, int axis, int quant_min, int quant_max) -> (Tensor output, Tensor mask)

self: fake_quantize_per_channel_affine_cachemask_backward(grad, mask)

- name: _fake_quantize_learnable_per_channel_affine(Tensor self, Tensor scale, Tensor zero_point, int axis, int quant_min, int quant_max, float grad_factor=1.0) -> Tensor

self, scale, zero_point: "grad.defined() ? _fake_quantize_learnable_per_channel_affine_backward(grad, self, scale, zero_point, axis, quant_min, quant_max, grad_factor) : std::tuple<Tensor, Tensor, Tensor>()"

- name: _fused_moving_avg_obs_fq_helper(Tensor self, Tensor observer_on, Tensor fake_quant_on, Tensor(a!) running_min, Tensor(b!) running_max, Tensor(c!) scale, Tensor(d!) zero_point, float averaging_const, int quant_min, int quant_max, int ch_axis, bool per_row_fake_quant=False, bool symmetric_quant=False) -> (Tensor output, Tensor mask)

self: fake_quantize_per_tensor_affine_cachemask_backward(grad, mask)

- name: fill_.Scalar(Tensor(a!) self, Scalar value) -> Tensor(a!)

self: zeros_like(grad)

result: self_t.fill_(0)

- name: fill_.Tensor(Tensor(a!) self, Tensor value) -> Tensor(a!)

self: zeros_like(grad)

value: grad.sum()

result: self_t.fill_(value_t)

- name: floor(Tensor self) -> Tensor

self: zeros_like(grad)

result: auto_element_wise

- name: fmod.Scalar(Tensor self, Scalar other) -> Tensor

self: grad

- name: fmod.Tensor(Tensor self, Tensor other) -> Tensor

self: grad

other: 'not_implemented("fmod: other")'

- name: frac(Tensor self) -> Tensor

self: grad

result: self_t

- name: frexp.Tensor(Tensor self) -> (Tensor mantissa, Tensor exponent)

self: grad / exponent.exp2()

mantissa: self_t / exponent.exp2()

- name: gather(Tensor self, int dim, Tensor index, *, bool sparse_grad=False) -> Tensor

self: gather_backward(grad, self, dim, index, sparse_grad)

index: non_differentiable

result: auto_linear

- name: ge_.Scalar(Tensor(a!) self, Scalar other) -> Tensor(a!)

self: zeros_like(self)

- name: ge_.Tensor(Tensor(a!) self, Tensor other) -> Tensor(a!)

self: zeros_like(self)

other: zeros_like(other)

- name: geometric_(Tensor(a!) self, float p, *, Generator? generator=None) -> Tensor(a!)

self: zeros_like(grad)

- name: geqrf(Tensor self) -> (Tensor a, Tensor tau)

self: not_implemented("geqrf")

- name: indices(Tensor(a) self) -> Tensor(a)

output_differentiability: [False]

- name: _indices(Tensor(a) self) -> Tensor(a)

output_differentiability: [False]

- name: grid_sampler_2d(Tensor input, Tensor grid, int interpolation_mode, int padding_mode, bool align_corners) -> Tensor

input, grid: "grad.defined() ? grid_sampler_2d_backward(grad, input, grid, interpolation_mode, padding_mode, align_corners) : std::tuple<Tensor, Tensor>()"

- name: grid_sampler_3d(Tensor input, Tensor grid, int interpolation_mode, int padding_mode, bool align_corners) -> Tensor

input, grid: "grad.defined() ? grid_sampler_3d_backward(grad, input, grid, interpolation_mode, padding_mode, align_corners) : std::tuple<Tensor, Tensor>()"

# See NOTE [ grid_sample CPU fallback ]

- name: _grid_sampler_2d_cpu_fallback(Tensor input, Tensor grid, int interpolation_mode, int padding_mode, bool align_corners) -> Tensor

input, grid: "grad.defined() ? _grid_sampler_2d_cpu_fallback_backward(grad, input, grid, interpolation_mode, padding_mode, align_corners) : std::tuple<Tensor, Tensor>()"

- name: gt_.Scalar(Tensor(a!) self, Scalar other) -> Tensor(a!)

self: zeros_like(self)

- name: gt_.Tensor(Tensor(a!) self, Tensor other) -> Tensor(a!)

self: zeros_like(self)

other: zeros_like(other)

- name: hardsigmoid(Tensor self) -> Tensor

self: hardsigmoid_backward(grad, self)

- name: histc(Tensor self, int bins=100, Scalar min=0, Scalar max=0) -> Tensor

self: not_implemented("histc")

- name: hardswish(Tensor self) -> Tensor

self: hardswish_backward(grad, self)

result: auto_element_wise

- name: hypot(Tensor self, Tensor other) -> Tensor

self: grad * self / result

other: grad * other / result

result: self_t * self_p / result + other_t * other_p / result

- name: i0(Tensor self) -> Tensor

self: grad * at::special_i1(self)

- name: special_i0e(Tensor self) -> Tensor

self: grad * (at::special_i1e(self) - self.sgn() * result)

- name: special_i1(Tensor self) -> Tensor

self: i1_backward(grad, self, result)

- name: special_i1e(Tensor self) -> Tensor

self: i1e_backward(grad, self, result)

- name: igamma(Tensor self, Tensor other) -> Tensor

self: 'not_implemented("igamma: input")'

other: grad * exp((self - 1) * log(other) - other - lgamma(self))

- name: igammac(Tensor self, Tensor other) -> Tensor

self: 'not_implemented("igammac: input")'

other: -grad * exp((self - 1) * log(other) - other - lgamma(self))

- name: index.Tensor(Tensor self, Tensor?[] indices) -> Tensor

self: index_backward(grad.new_zeros(self.sizes(), self.options()), indices, grad)

- name: index_add_.alpha(Tensor(a!) self, int dim, Tensor index, Tensor source, *, Scalar alpha) -> Tensor(a!)

self: grad

# The case source.dim() == 0 is necessary to support scalar tensors of the form

# source.dim() == 0 and index.dim() == 1 and index.size() == (1,),

# This is because source is not broadcastable to index, as source.dim() < index.dim()

source: "maybe_multiply(source.dim() > 0 ? grad.index_select(dim, index).expand_as(source) : grad.index_select(dim, index.squeeze(0)), alpha)"

index: non_differentiable

result: self_t.index_add_(dim, index, maybe_multiply(source_t, alpha))

- name: index_copy_(Tensor(a!) self, int dim, Tensor index, Tensor source) -> Tensor(a!)

self: grad.clone().index_fill_(dim, index, 0)

# The case source.dim() == 0 is necessary to support scalar tensors of the form

# source.dim() == 0 and index.dim() == 1 and index.size() == (1,),

# This is because source is not broadcastable to index, as source.dim() < index.dim()

source: "source.dim() > 0 ? grad.index_select(dim, index).expand_as(source) : grad.index_select(dim, index.squeeze(0))"

index: non_differentiable

result: self_t.index_copy_(dim, index, source_t)

- name: index_fill_.int_Scalar(Tensor(a!) self, int dim, Tensor index, Scalar value) -> Tensor(a!)

self: grad.clone().index_fill_(dim, index, 0)

index: non_differentiable

result: self_t.index_fill_(dim, index, 0)

- name: index_fill_.int_Tensor(Tensor(a!) self, int dim, Tensor index, Tensor value) -> Tensor(a!)

self: grad.clone().index_fill_(dim, index, 0)

value: grad.index_select(dim, std::get<0>(at::_unique(index, /*sorted=*/false))).sum()

index: non_differentiable

result: self_t.index_fill_(dim, index, value_t)

- name: index_put_(Tensor(a!) self, Tensor?[] indices, Tensor values, bool accumulate=False) -> Tensor(a!)

self: "accumulate ? grad : grad.clone().index_put_(indices, zeros_like(values), false)"

values: grad.index(indices)

result: self_t.index_put_(indices, values_t, accumulate)

- name: _index_put_impl_(Tensor(a!) self, Tensor?[] indices, Tensor values, bool accumulate=False, bool unsafe=False) -> Tensor(a!)

self: "accumulate ? grad : grad.clone().index_put_(indices, zeros_like(values), false)"

values: grad.index(indices)

result: at::_index_put_impl_(self_t, indices, values_t, accumulate, unsafe)

- name: index_select(Tensor self, int dim, Tensor index) -> Tensor

self: index_select_backward(grad, self.sizes(), dim, index)

index: non_differentiable

result: auto_linear

- name: inverse(Tensor self) -> Tensor

self: -at::matmul(result.conj().transpose(-2, -1), at::matmul(grad, result.conj().transpose(-2, -1)))

- name: linalg_inv_ex(Tensor self, *, bool check_errors=False) -> (Tensor inverse, Tensor info)

self: -at::matmul(inverse.conj().transpose(-2, -1), at::matmul(grad, inverse.conj().transpose(-2, -1)))

inverse: -at::matmul(at::matmul(inverse, self_t), inverse)

- name: isnan(Tensor self) -> Tensor

self: non_differentiable

- name: kthvalue(Tensor self, int k, int dim=-1, bool keepdim=False) -> (Tensor values, Tensor indices)

self: value_selecting_reduction_backward(grad, dim, indices, self.sizes(), keepdim)

values: gather_with_keepdimed_indices(self_t, dim, indices, keepdim)

- name: le_.Scalar(Tensor(a!) self, Scalar other) -> Tensor(a!)

self: zeros_like(self)

- name: le_.Tensor(Tensor(a!) self, Tensor other) -> Tensor(a!)

self: zeros_like(self)

other: zeros_like(other)

- name: lerp.Scalar(Tensor self, Tensor end, Scalar weight) -> Tensor

self: "weight.isComplex() ? grad * (1 - weight.conj().toComplexDouble()) : grad * (1 - weight.toDouble())"

end: grad * weight.conj()

result: at::lerp(self_t, end_t, weight)

- name: lerp.Tensor(Tensor self, Tensor end, Tensor weight) -> Tensor

self: grad * (1 - weight).conj()

end: grad * weight.conj()

weight: grad * (end - self).conj()

result: at::lerp(self_t, end_t, weight_p) + weight_t * (end_p - self_p)

- name: lgamma(Tensor self) -> Tensor

self: grad * digamma(self)

result: auto_element_wise

- name: digamma(Tensor self) -> Tensor

self: grad * polygamma(1, self)

result: auto_element_wise

- name: polygamma(int n, Tensor self) -> Tensor

self: grad * polygamma(n + 1, self)

result: auto_element_wise

- name: polygamma_(Tensor(a!) self, int n) -> Tensor(a!)

self: grad * polygamma(n + 1, self)

result: self_t.mul_(polygamma(n + 1, original_self_p))

- name: log(Tensor self) -> Tensor

self: grad.div(self.conj())

result: auto_element_wise

- name: log10(Tensor self) -> Tensor

self: grad / (self.conj() * 2.3025850929940456)

result: auto_element_wise

- name: log1p(Tensor self) -> Tensor

self: log1p_backward(grad, self)

result: auto_element_wise

- name: log2(Tensor self) -> Tensor

self: grad / (self.conj() * 0.6931471805599453)

result: auto_element_wise

- name: logaddexp(Tensor self, Tensor other) -> Tensor

self: grad / (1 + exp(other - self))

other: grad / (1 + exp(self - other))

result: self_t / (1 + exp(other_p - self_p)) + other_t / (1 + exp(self_p - other_p))

- name: logaddexp2(Tensor self, Tensor other) -> Tensor

self: grad / (1 + pow(2, other - self))

other: grad / (1 + pow(2, self - other))

result: self_t / (1 + pow(2, other_p - self_p)) + other_t / (1 + pow(2, self_p - other_p))

- name: xlogy.Tensor(Tensor self, Tensor other) -> Tensor

self: grad * at::xlogy((self != 0), other)

other: grad * at::where(other.isnan() | (self != 0), self / other, zeros_like(other))

result: self_t * at::xlogy((self_p != 0), other_p) + other_t * self_p / other_p

- name: xlogy.Scalar_Self(Scalar self, Tensor other) -> Tensor

other: grad * at::where(other.isnan() | (!self.equal(0)), self / other, zeros_like(other))

result: auto_element_wise

- name: xlogy.Scalar_Other(Tensor self, Scalar other) -> Tensor

self: grad * at::xlogy((self != 0), other)

result: auto_element_wise

- name: special_xlog1py(Tensor self, Tensor other) -> Tensor

self: grad * other.log1p()

other: grad * self / (other + 1)

result: self_t * other_p.log1p() + other_t * self_p / (other_p + 1)

- name: special_xlog1py.self_scalar(Scalar self, Tensor other) -> Tensor

other: grad * self / (other + 1)

result: auto_element_wise

- name: special_xlog1py.other_scalar(Tensor self, Scalar other) -> Tensor

self: grad * log1p(other.toDouble())

result: auto_element_wise

- name: special_zeta(Tensor self, Tensor other) -> Tensor

self: not_implemented("zeta")

other: grad * -self * special_zeta(self + 1., other)

- name: special_zeta.self_scalar(Scalar self, Tensor other) -> Tensor

other: grad * -self * special_zeta(self.toDouble() + 1., other)

- name: special_zeta.other_scalar(Tensor self, Scalar other) -> Tensor

self: not_implemented("zeta")

- name: logdet(Tensor self) -> Tensor

self: logdet_backward(grad, self, result)

- name: log_normal_(Tensor(a!) self, float mean=1, float std=2, *, Generator? generator=None) -> Tensor(a!)

self: zeros_like(grad)

- name: logsumexp(Tensor self, int[1] dim, bool keepdim=False) -> Tensor

self: logsumexp_backward(grad, self, result, dim, keepdim)

- name: lstsq(Tensor self, Tensor A) -> (Tensor solution, Tensor QR)

self: not_implemented("lstsq")

A: not_implemented("lstsq")

- name: linalg_lstsq(Tensor self, Tensor b, float? rcond=None, *, str? driver=None) -> (Tensor solution, Tensor residuals, Tensor rank, Tensor singular_values)

self: not_implemented("linalg_lstsq")

b: not_implemented("linalg_lstsq")

output_differentiability: [True, True]

- name: lt_.Scalar(Tensor(a!) self, Scalar other) -> Tensor(a!)

self: zeros_like(self)

- name: lt_.Tensor(Tensor(a!) self, Tensor other) -> Tensor(a!)

self: zeros_like(self)

other: zeros_like(other)

- name: _lu_with_info(Tensor self, bool pivot=True, bool check_errors=True) -> (Tensor LU, Tensor pivots, Tensor info)

self: _lu_with_info_backward(grad, self, LU, pivots)

LU: _lu_with_info_jvp(self_t, LU, pivots)

- name: lu_solve(Tensor self, Tensor LU_data, Tensor LU_pivots) -> Tensor

self, LU_data: lu_solve_backward(grad, self, LU_data, LU_pivots)

result: lu_solve_forward_AD(self_t, LU_data_t, LU_data_p, LU_pivots, result)

- name: lu_unpack(Tensor LU_data, Tensor LU_pivots, bool unpack_data=True, bool unpack_pivots=True) -> (Tensor P, Tensor L, Tensor U)

LU_data: lu_unpack_backward(grads, LU_data, unpack_data)

LU_pivots: non_differentiable

- name: masked_fill_.Scalar(Tensor(a!) self, Tensor mask, Scalar value) -> Tensor(a!)

self: grad.clone().masked_fill_(mask, 0)

mask: non_differentiable

result: self_t.masked_fill_(mask, 0)

- name: masked_fill_.Tensor(Tensor(a!) self, Tensor mask, Tensor value) -> Tensor(a!)

self: grad.clone().masked_fill_(mask, 0)

value: at::where(mask, grad, zeros_like(grad)).sum()

mask: non_differentiable

result: self_t.masked_fill_(mask, value_t)

- name: masked_scatter_(Tensor(a!) self, Tensor mask, Tensor source) -> Tensor(a!)

self: grad.clone().masked_fill_(mask, 0)

source: masked_scatter_backward(grad, mask, source.sizes())

mask: non_differentiable

result: self_t.masked_scatter_(mask, source_t)

- name: masked_select(Tensor self, Tensor mask) -> Tensor

self: masked_select_backward(grad, self, mask)

mask: non_differentiable

result: auto_linear

- name: matrix_exp(Tensor self) -> Tensor

self: matrix_exp_backward(self, grad)

- name: max.dim(Tensor self, int dim, bool keepdim=False) -> (Tensor values, Tensor indices)

self: value_selecting_reduction_backward(grad, dim, indices, self.sizes(), keepdim)

values: gather_with_keepdimed_indices(self_t, dim, indices, keepdim)

- name: max(Tensor self) -> Tensor

self: evenly_distribute_backward(grad, self, result)

result: evenly_read_jvp(self_t, self_p, result)

- name: maximum(Tensor self, Tensor other) -> Tensor

self: at::where(self == other, grad / 2, grad).masked_fill_(self < other, 0)

other: at::where(self == other, grad / 2, grad).masked_fill_(self > other, 0)

result: other_t + at::where(self_p == other_p, 0.5, (self_p > other_p).to(result.scalar_type())) * (self_t - other_t)

- name: fmax(Tensor self, Tensor other) -> Tensor

self: grad.clone().masked_fill_((self >= other).logical_or_(other.isnan()).logical_not_(), 0)

other: grad.clone().masked_fill_((self >= other).logical_or_(other.isnan()), 0)

result: other_t + (self_p > other_p).logical_or_(other_p.isnan()) * (self_t - other_t)

- name: mean(Tensor self, *, ScalarType? dtype=None) -> Tensor

self: grad.expand(self.sizes()).to(self.scalar_type()) / self.numel()

result: auto_linear

- name: mean.dim(Tensor self, int[1] dim, bool keepdim=False, *, ScalarType? dtype=None) -> Tensor

self: sum_backward(grad, self.sizes(), dim, keepdim).to(self.scalar_type()) / _safe_size(self.sizes(), dim)

result: auto_linear

- name: median(Tensor self) -> Tensor

self: evenly_distribute_backward(grad, self, result)

- name: nanmedian(Tensor self) -> Tensor

self: evenly_distribute_backward(grad, self, result)

# This is in theory incorrect in the following case:

# sorted list: [..., a, b, b, ..., b, b, c, ...] with median = b and the value

# | at middle position of the

# | list between two `b`s. E.g.,

# |

# ^the middle position

# The gradient exists and is essentially 0 in this case.

#

# In case where the middle position is at the boundary of `b` range, e.g.,

# sorted list: [..., a, b, b, ..., b, b, c, ...]

# |

# ^the middle position

# The backward implementation is correct in the sense that it returns the

# subgradient on one side.

- name: median.dim(Tensor self, int dim, bool keepdim=False) -> (Tensor values, Tensor indices)

self: value_selecting_reduction_backward(grad, dim, indices, self.sizes(), keepdim)

- name: nanmedian.dim(Tensor self, int dim, bool keepdim=False) -> (Tensor values, Tensor indices)

self: value_selecting_reduction_backward(grad, dim, indices, self.sizes(), keepdim)

- name: min.dim(Tensor self, int dim, bool keepdim=False) -> (Tensor values, Tensor indices)

self: value_selecting_reduction_backward(grad, dim, indices, self.sizes(), keepdim)

values: gather_with_keepdimed_indices(self_t, dim, indices, keepdim)

- name: min(Tensor self) -> Tensor

self: evenly_distribute_backward(grad, self, result)

result: evenly_read_jvp(self_t, self_p, result)

- name: minimum(Tensor self, Tensor other) -> Tensor

self: at::where(self == other, grad / 2, grad).masked_fill_(self > other, 0)

other: at::where(self == other, grad / 2, grad).masked_fill_(self < other, 0)

result: other_t + at::where(self_p == other_p, 0.5, (self_p < other_p).to(result.scalar_type())) * (self_t - other_t)

- name: fmin(Tensor self, Tensor other) -> Tensor

self: grad.clone().masked_fill_((self <= other).logical_or_(other.isnan()).logical_not_(), 0)

other: grad.clone().masked_fill_((self <= other).logical_or_(other.isnan()), 0)

result: other_t + (self_p <= other_p).logical_or_(other_p.isnan()) * (self_t - other_t)

- name: amax(Tensor self, int[1] dim=[], bool keepdim=False) -> Tensor

self: scale_grad_by_count(restore_reduced_dims(grad, dim, keepdim), restore_reduced_dims(result, dim, keepdim) == self, dim)

- name: amin(Tensor self, int[1] dim=[], bool keepdim=False) -> Tensor

self: scale_grad_by_count(restore_reduced_dims(grad, dim, keepdim), restore_reduced_dims(result, dim, keepdim) == self, dim)

- name: mm(Tensor self, Tensor mat2) -> Tensor

self: mm_mat1_backward(grad, mat2, self.sizes(), self.strides(), 1)

mat2: mm_mat2_backward(grad, self, mat2.sizes(), mat2.strides(), 1)

result: at::mm(self_t, mat2_p) + at::mm(self_p, mat2_t)

- name: mode(Tensor self, int dim=-1, bool keepdim=False) -> (Tensor values, Tensor indices)

self: value_selecting_reduction_backward(grad, dim, indices, self.sizes(), keepdim)

values: gather_with_keepdimed_indices(self_t, dim, indices, keepdim)

- name: mul.Tensor(Tensor self, Tensor other) -> Tensor

self: mul_tensor_backward(grad, other, self.scalar_type())

other: mul_tensor_backward(grad, self, other.scalar_type())

result: other_t * self_p + self_t * other_p

- name: mul.Scalar(Tensor self, Scalar other) -> Tensor

self: mul_tensor_backward(grad, at::scalar_to_tensor(other), self.scalar_type())

result: self_t * other