作为一个张量计算库，算数操作是最基本的修养。PyTorch 中有一百来个算数操作算子。
逐元素操作算子PyTorch 中，Tensor 的逐元素操作可分为函数式操作和原位操作，在函数名中用 _ 后缀区分了二者，比如 tensor.abs() 是一个函数式操作，不改变 tensor 而将改变体现在返回值中，而 tensor.abs_() 是一个原位操作，既修改了原 tensor 又将这个修改后的值返回。
这里的操作都是逐元素进行的，元素之间没有关联。基本上每个 API 都有函数式和原位两个版本，也有作为 torch 模块的函数的过程化编程和作为 Tensor 的方法的面向对象式编程，其实面向过程和面向对象的唯一区别是第一个参数。每个算子一共有至少三种写法。
| 算子名 | 说明 |
|---|---|
| abs | 计算 input 中每个元素的绝对值. |
| absolute | Alias for torch.abs() |
| acos | inverse cosine 大家熟悉的 arccos input. |
| arccos | Alias for torch.acos(). |
| acosh | inverse hyperbolic cosine of input. |
| arccosh | Alias for torch.acosh(). |
| add | Adds other, scaled by alpha, to input. |
| addcdiv | Performs the element-wise division of tensor1 by tensor2, multiply the result by the scalar value and add it to input. |
| addcmul | Performs the element-wise multiplication of tensor1 by tensor2, multiply the result by the scalar value and add it to input. |
| angle | Computes the element-wise angle (in radians) of the given input tensor. |
| asin | Returns a new tensor with the arcsine of the elements of input. |
| arcsin | Alias for torch.asin(). |
| asinh | Returns a new tensor with the inverse hyperbolic sine of the elements of input. |
| arcsinh | Alias for torch.asinh(). |
| atan | Returns a new tensor with the arctangent of the elements of input. |
| arctan | Alias for torch.atan(). |
| atanh | Returns a new tensor with the inverse hyperbolic tangent of the elements of input. |
| arctanh | Alias for torch.atanh(). |
| atan2 | Element-wise arctangent of text{input}{i} / text{other}{i}inputi/otheri with consideration of the quadrant. |
| arctan2 | Alias for torch.atan2(). |
| bitwise_not | Computes the bitwise NOT of the given input tensor. |
| bitwise_and | Computes the bitwise AND of input and other. |
| bitwise_or | Computes the bitwise OR of input and other. |
| bitwise_xor | Computes the bitwise XOR of input and other. |
| bitwise_left_shift | Computes the left arithmetic shift of input by other bits. |
| bitwise_right_shift | Computes the right arithmetic shift of input by other bits. |
| ceil | Returns a new tensor with the ceil of the elements of input, the smallest integer greater than or equal to each element. |
| clamp | Clamps all elements in input into the range [ min, max ]. |
| clip | Alias for torch.clamp(). |
| conj_physical | Computes the element-wise conjugate of the given input tensor. |
| copysign | Create a new floating-point tensor with the magnitude of input and the sign of other, elementwise. |
| cos | Returns a new tensor with the cosine of the elements of input. |
| cosh | Returns a new tensor with the hyperbolic cosine of the elements of input. |
| deg2rad | Returns a new tensor with each of the elements of input converted from angles in degrees to radians. |
| div | Divides each element of the input input by the corresponding element of other. |
| divide | Alias for torch.div(). |
| digamma | Alias for torch.special.digamma(). |
| erf | Alias for torch.special.erf(). |
| erfc | Alias for torch.special.erfc(). |
| erfinv | Alias for torch.special.erfinv(). |
| exp | Returns a new tensor with the exponential of the elements of the input tensor input. |
| exp2 | Alias for torch.special.exp2(). |
| expm1 | Alias for torch.special.expm1(). |
| fake_quantize_per_channel_affine | Returns a new tensor with the data in input fake quantized per channel using scale, zero_point, quant_min and quant_max, across the channel specified by axis. |
| fake_quantize_per_tensor_affine | Returns a new tensor with the data in input fake quantized using scale, zero_point, quant_min and quant_max. |
| fix | Alias for torch.trunc() |
| float_power | Raises input to the power of exponent, elementwise, in double precision. |
| floor | Returns a new tensor with the floor of the elements of input, the largest integer less than or equal to each element. |
| floor_divide | |
| fmod | Applies C++’s std::fmod entrywise. |
| frac | Computes the fractional portion of each element in input. |
| frexp | Decomposes input into mantissa and exponent tensors such that input = mantissa × 2 exponent text{input} = text{mantissa} times 2^{text{exponent}} input=mantissa×2exponent. |
| gradient | Estimates the gradient of a function g : R n → R g : mathbb{R}^n rightarrow mathbb{R} g:Rn→R in one or more dimensions using the second-order accurate central differences method. |
| imag | Returns a new tensor containing imaginary values of the self tensor. |
| ldexp | Multiplies input by 2 other 2^text{other} 2other. |
| lerp | Does a linear interpolation of two tensors start (given by input) and end based on a scalar or tensor weight and returns the resulting out tensor. |
| lgamma | Computes the natural logarithm of the absolute value of the gamma function on input. |
| log | Returns a new tensor with the natural logarithm of the elements of input. |
| log10 | Returns a new tensor with the logarithm to the base 10 of the elements of input. |
| log1p | Returns a new tensor with the natural logarithm of (1 + input). |
| log2 | Returns a new tensor with the logarithm to the base 2 of the elements of input. |
| logaddexp | Logarithm of the sum of exponentiations of the inputs. |
| logaddexp2 | Logarithm of the sum of exponentiations of the inputs in base-2. |
| logical_and | Computes the element-wise logical AND of the given input tensors. |
| logical_not | Computes the element-wise logical NOT of the given input tensor. |
| logical_or | Computes the element-wise logical OR of the given input tensors. |
| logical_xor | Computes the element-wise logical XOR of the given input tensors. |
| logit | Alias for torch.special.logit(). |
| hypot | Given the legs of a right triangle, return its hypotenuse. |
| i0 | Alias for torch.special.i0(). |
| igamma | Alias for torch.special.gammainc(). |
| igammac | Alias for torch.special.gammaincc(). |
| mul | Multiplies input by other. |
| multiply | Alias for torch.mul(). |
| mvlgamma | Alias for torch.special.multigammaln(). |
| nan_to_num | Replaces NaN, positive infinity, and negative infinity values in input with the values specified by nan, posinf, and neginf, respectively. |
| neg | Returns a new tensor with the negative of the elements of input. |
| negative | Alias for torch.neg() |
| nextafter | Return the next floating-point value after input towards other, elementwise. |
| polygamma | Alias for torch.special.polygamma(). |
| positive | Returns input. |
| pow | Takes the power of each element in input with exponent and returns a tensor with the result. |
| quantized_batch_norm | Applies batch normalization on a 4D (NCHW) quantized tensor. |
| quantized_max_pool1d | Applies a 1D max pooling over an input quantized tensor composed of several input planes. |
| quantized_max_pool2d | Applies a 2D max pooling over an input quantized tensor composed of several input planes. |
| rad2deg | Returns a new tensor with each of the elements of input converted from angles in radians to degrees. |
| real | Returns a new tensor containing real values of the self tensor. |
| reciprocal | Returns a new tensor with the reciprocal of the elements of input |
| remainder | Computes Python’s modulus operation entrywise. |
| round | Rounds elements of input to the nearest integer. |
| rsqrt | Returns a new tensor with the reciprocal of the square-root of each of the elements of input. |
| sigmoid | Alias for torch.special.expit(). |
| sign | Returns a new tensor with the signs of the elements of input. |
| sgn | This function is an extension of torch.sign() to complex tensors. |
| signbit | Tests if each element of input has its sign bit set (is less than zero) or not. |
| sin | Returns a new tensor with the sine of the elements of input. |
| sinc | Alias for torch.special.sinc(). |
| sinh | Returns a new tensor with the hyperbolic sine of the elements of input. |
| sqrt | Returns a new tensor with the square-root of the elements of input. |
| square | Returns a new tensor with the square of the elements of input. |
| sub | Subtracts other, scaled by alpha, from input. |
| subtract | Alias for torch.sub(). |
| tan | Returns a new tensor with the tangent of the elements of input. |
| tanh | Returns a new tensor with the hyperbolic tangent of the elements of input. |
| true_divide | Alias for torch.div() with rounding_mode=None. |
| trunc | Returns a new tensor with the truncated integer values of the elements of input. |
| xlogy | Alias for torch.special.xlogy(). |
除了上述函数，PyTorch 还为我们重载了 Tensor 的很多运算符，这些运算符都符合 Python 的语义。PyTorch 的开发者为了 be Pythonic 还是费了相当大心思的，这也是 PyTorch 能很好融入 Python 生态的原因。
| 运算符 | 说明 |
|---|---|
| + | add |
| - | sub |
| * | mul |
| / | div |
| // | truediv |
| @ | matmul |
规约是函数式编程中的重要概念。
| 算子名 | 说明 |
|---|---|
| argmax | Returns the indices of the maximum value of all elements in the input tensor. |
| argmin | Returns the indices of the minimum value(s) of the flattened tensor or along a dimension |
| amax | Returns the maximum value of each slice of the input tensor in the given dimension(s) dim. |
| amin | Returns the minimum value of each slice of the input tensor in the given dimension(s) dim. |
| aminmax | Computes the minimum and maximum values of the input tensor. |
| all | Tests if all elements in input evaluate to True. |
| any | Tests if any element in input evaluates to True. |
| max | Returns the maximum value of all elements in the input tensor. |
| min | Returns the minimum value of all elements in the input tensor. |
| dist | Returns the p-norm of (input - other) |
| logsumexp | Returns the log of summed exponentials of each row of the input tensor in the given dimension dim. |
| mean | Returns the mean value of all elements in the input tensor. |
| nanmean | Computes the mean of all non-NaN elements along the specified dimensions. |
| median | Returns the median of the values in input. |
| nanmedian | Returns the median of the values in input, ignoring NaN values. |
| mode | Returns a namedtuple (values, indices) where values is the mode value of each row of the input tensor in the given dimension dim, i.e. a value which appears most often in that row, and indices is the index location of each mode value found. |
| norm | Returns the matrix norm or vector norm of a given tensor. |
| nansum | Returns the sum of all elements, treating Not a Numbers (NaNs) as zero. |
| prod | Returns the product of all elements in the input tensor. |
| quantile | Computes the q-th quantiles of each row of the input tensor along the dimension dim. |
| nanquantile | This is a variant of torch.quantile() that “ignores” NaN values, computing the quantiles q as if NaN values in input did not exist. |
| std | If unbiased is True, Bessel’s correction will be used. |
| std_mean | If unbiased is True, Bessel’s correction will be used to calculate the standard deviation. |
| sum | Returns the sum of all elements in the input tensor. |
| unique | Returns the unique elements of the input tensor. |
| unique_consecutive | Eliminates all but the first element from every consecutive group of equivalent elements. |
| var | If unbiased is True, Bessel’s correction will be used. |
| var_mean | If unbiased is True, Bessel’s correction will be used to calculate the variance. |
| count_nonzero | Counts the number of non-zero values in the tensor input along the given dim. |
| allclose | This function checks if all input and other satisfy the condition: |
|---|---|
| argsort | Returns the indices that sort a tensor along a given dimension in ascending order by value. |
| eq | Computes element-wise equality |
| equal | True if two tensors have the same size and elements, False otherwise. |
| ge | Computes text{input} geq text{other}input≥other element-wise. |
| greater_equal | Alias for torch.ge(). |
| gt | Computes text{input} > text{other}input>other element-wise. |
| greater | Alias for torch.gt(). |
| isclose | Returns a new tensor with boolean elements representing if each element of input is “close” to the corresponding element of other. |
| isfinite | Returns a new tensor with boolean elements representing if each element is finite or not. |
| isin | Tests if each element of elements is in test_elements. |
| isinf | Tests if each element of input is infinite (positive or negative infinity) or not. |
| isposinf | Tests if each element of input is positive infinity or not. |
| isneginf | Tests if each element of input is negative infinity or not. |
| isnan | Returns a new tensor with boolean elements representing if each element of input is NaN or not. |
| isreal | Returns a new tensor with boolean elements representing if each element of input is real-valued or not. |
| kthvalue | Returns a namedtuple (values, indices) where values is the k th smallest element of each row of the input tensor in the given dimension dim. |
| le | Computes text{input} leq text{other}input≤other element-wise. |
| less_equal | Alias for torch.le(). |
| lt | Computes text{input} < text{other}input |
| less | Alias for torch.lt(). |
| maximum | Computes the element-wise maximum of input and other. |
| minimum | Computes the element-wise minimum of input and other. |
| fmax | Computes the element-wise maximum of input and other. |
| fmin | Computes the element-wise minimum of input and other. |
| ne | Computes text{input} neq text{other}input=other element-wise. |
| not_equal | Alias for torch.ne(). |
| sort | Sorts the elements of the input tensor along a given dimension in ascending order by value. |
| topk | Returns the k largest elements of the given input tensor along a given dimension. |
| msort | Sorts the elements of the input tensor along its first dimension in ascending order by value. |
| stft | Short-time Fourier transform (STFT). |
|---|---|
| istft | Inverse short time Fourier Transform. |
| bartlett_window | Bartlett window function. |
| blackman_window | Blackman window function. |
| hamming_window | Hamming window function. |
| hann_window | Hann window function. |
| kaiser_window | Computes the Kaiser window with window length window_length and shape parameter beta. |
| atleast_1d | Returns a 1-dimensional view of each input tensor with zero dimensions. |
|---|---|
| atleast_2d | Returns a 2-dimensional view of each input tensor with zero dimensions. |
| atleast_3d | Returns a 3-dimensional view of each input tensor with zero dimensions. |
| bincount | Count the frequency of each value in an array of non-negative ints. |
| block_diag | Create a block diagonal matrix from provided tensors. |
| broadcast_tensors | Broadcasts the given tensors according to Broadcasting semantics. |
| broadcast_to | Broadcasts input to the shape shape. |
| broadcast_shapes | Similar to broadcast_tensors() but for shapes. |
| bucketize | Returns the indices of the buckets to which each value in the input belongs, where the boundaries of the buckets are set by boundaries. |
| cartesian_prod | Do cartesian product of the given sequence of tensors. |
| cdist | Computes batched the p-norm distance between each pair of the two collections of row vectors. |
| clone | Returns a copy of input. |
| combinations | Compute combinations of length rr of the given tensor. |
| corrcoef | Estimates the Pearson product-moment correlation coefficient matrix of the variables given by the input matrix, where rows are the variables and columns are the observations. |
| cov | Estimates the covariance matrix of the variables given by the input matrix, where rows are the variables and columns are the observations. |
| cross | Returns the cross product of vectors in dimension dim of input and other. |
| cummax | Returns a namedtuple (values, indices) where values is the cumulative maximum of elements of input in the dimension dim. |
| cummin | Returns a namedtuple (values, indices) where values is the cumulative minimum of elements of input in the dimension dim. |
| cumprod | Returns the cumulative product of elements of input in the dimension dim. |
| cumsum | Returns the cumulative sum of elements of input in the dimension dim. |
| diag | If input is a vector (1-D tensor), then returns a 2-D square tensor |
| diag_embed | Creates a tensor whose diagonals of certain 2D planes (specified by dim1 and dim2) are filled by input. |
| diagflat | If input is a vector (1-D tensor), then returns a 2-D square tensor |
| diagonal | Returns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape. |
| diff | Computes the n-th forward difference along the given dimension. |
| einsum | Sums the product of the elements of the input operands along dimensions specified using a notation based on the Einstein summation convention. |
| flatten | Flattens input by reshaping it into a one-dimensional tensor. |
| flip | Reverse the order of a n-D tensor along given axis in dims. |
| fliplr | Flip tensor in the left/right direction, returning a new tensor. |
| flipud | Flip tensor in the up/down direction, returning a new tensor. |
| kron | Computes the Kronecker product, denoted by otimes⊗, of input and other. |
| rot90 | Rotate a n-D tensor by 90 degrees in the plane specified by dims axis. |
| gcd | Computes the element-wise greatest common divisor (GCD) of input and other. |
| histc | Computes the histogram of a tensor. |
| histogram | Computes a histogram of the values in a tensor. |
| histogramdd | Computes a multi-dimensional histogram of the values in a tensor. |
| meshgrid | Creates grids of coordinates specified by the 1D inputs in attr:tensors. |
| lcm | Computes the element-wise least common multiple (LCM) of input and other. |
| logcumsumexp | Returns the logarithm of the cumulative summation of the exponentiation of elements of input in the dimension dim. |
| ravel | Return a contiguous flattened tensor. |
| renorm | Returns a tensor where each sub-tensor of input along dimension dim is normalized such that the p-norm of the sub-tensor is lower than the value maxnorm |
| repeat_interleave | Repeat elements of a tensor. |
| roll | Roll the tensor input along the given dimension(s). |
| searchsorted | Find the indices from the innermost dimension of sorted_sequence such that, if the corresponding values in values were inserted before the indices, when sorted, the order of the corresponding innermost dimension within sorted_sequence would be preserved. |
| tensordot | Returns a contraction of a and b over multiple dimensions. |
| trace | Returns the sum of the elements of the diagonal of the input 2-D matrix. |
| tril | Returns the lower triangular part of the matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0. |
| tril_indices | Returns the indices of the lower triangular part of a row-by- col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates. |
| triu | Returns the upper triangular part of a matrix (2-D tensor) or batch of matrices input, the other elements of the result tensor out are set to 0. |
| triu_indices | Returns the indices of the upper triangular part of a row by col matrix in a 2-by-N Tensor, where the first row contains row coordinates of all indices and the second row contains column coordinates. |
| vander | Generates a Vandermonde matrix. |
| view_as_real | Returns a view of input as a real tensor. |
| view_as_complex | Returns a view of input as a complex tensor. |
| resolve_conj | Returns a new tensor with materialized conjugation if input’s conjugate bit is set to True, else returns input. |
| resolve_neg | Returns a new tensor with materialized negation if input’s negative bit is set to True, else returns input. |
| addbmm | Performs a batch matrix-matrix product of matrices stored in batch1 and batch2, with a reduced add step (all matrix multiplications get accumulated along the first dimension). |
|---|---|
| addmm | Performs a matrix multiplication of the matrices mat1 and mat2. |
| addmv | Performs a matrix-vector product of the matrix mat and the vector vec. |
| addr | Performs the outer-product of vectors vec1 and vec2 and adds it to the matrix input. |
| baddbmm | Performs a batch matrix-matrix product of matrices in batch1 and batch2. |
| bmm | Performs a batch matrix-matrix product of matrices stored in input and mat2. |
| chain_matmul | Returns the matrix product of the NN 2-D tensors. |
| cholesky | Computes the Cholesky decomposition of a symmetric positive-definite matrix AA or for batches of symmetric positive-definite matrices. |
| cholesky_inverse | Computes the inverse of a symmetric positive-definite matrix AA using its Cholesky factor uu: returns matrix inv. |
| cholesky_solve | Solves a linear system of equations with a positive semidefinite matrix to be inverted given its Cholesky factor matrix uu. |
| dot | Computes the dot product of two 1D tensors. |
| eig | Computes the eigenvalues and eigenvectors of a real square matrix. |
| geqrf | This is a low-level function for calling LAPACK’s geqrf directly. |
| ger | Alias of torch.outer(). |
| inner | Computes the dot product for 1D tensors. |
| inverse | Alias for torch.linalg.inv() |
| det | Alias for torch.linalg.det() |
| logdet | Calculates log determinant of a square matrix or batches of square matrices. |
| slogdet | Alias for torch.linalg.slogdet() |
| lstsq | Computes the solution to the least squares and least norm problems for a full rank matrix AA of size (m times n)(m×n) and a matrix BB of size (m times k)(m×k). |
| lu | Computes the LU factorization of a matrix or batches of matrices A. |
| lu_solve | Returns the LU solve of the linear system Ax = bA**x=b using the partially pivoted LU factorization of A from torch.lu(). |
| lu_unpack | Unpacks the data and pivots from a LU factorization of a tensor into tensors L and U and a permutation tensor P such that LU_data, LU_pivots = (P @ L @ U).lu(). |
| matmul | Matrix product of two tensors. |
| matrix_power | Alias for torch.linalg.matrix_power() |
| matrix_rank | Returns the numerical rank of a 2-D tensor. |
| matrix_exp | Alias for torch.linalg.matrix_exp(). |
| mm | Performs a matrix multiplication of the matrices input and mat2. |
| mv | Performs a matrix-vector product of the matrix input and the vector vec. |
| orgqr | Alias for torch.linalg.householder_product(). |
| ormqr | Computes the matrix-matrix multiplication of a product of Householder matrices with a general matrix. |
| outer | Outer product of input and vec2. |
| pinverse | Alias for torch.linalg.pinv() |
| qr | Computes the QR decomposition of a matrix or a batch of matrices input, and returns a namedtuple (Q, R) of tensors such that text{input} = Q Rinput=QR with QQ being an orthogonal matrix or batch of orthogonal matrices and RR being an upper triangular matrix or batch of upper triangular matrices. |
| svd | Computes the singular value decomposition of either a matrix or batch of matrices input. |
| svd_lowrank | Return the singular value decomposition (U, S, V) of a matrix, batches of matrices, or a sparse matrix AA such that A approx U diag(S) V^TA≈Udia**g(S)V**T. |
| pca_lowrank | Performs linear Principal Component Analysis (PCA) on a low-rank matrix, batches of such matrices, or sparse matrix. |
| symeig | This function returns eigenvalues and eigenvectors of a real symmetric or complex Hermitian matrix input or a batch thereof, represented by a namedtuple (eigenvalues, eigenvectors). |
| lobpcg | Find the k largest (or smallest) eigenvalues and the corresponding eigenvectors of a symmetric positive definite generalized eigenvalue problem using matrix-free LOBPCG methods. |
| trapz | Alias for torch.trapezoid(). |
| trapezoid | Computes the trapezoidal rule along dim. |
| cumulative_trapezoid | Cumulatively computes the trapezoidal rule along dim. |
| triangular_solve | Solves a system of equations with a square upper or lower triangular invertible matrix AA and multiple right-hand sides bb. |
| vdot | Computes the dot product of two 1D tensors. |
| compiled_with_cxx11_abi | Returns whether PyTorch was built with _GLIBCXX_USE_CXX11_ABI=1 |
|---|---|
| result_type | Returns the torch.dtype that would result from performing an arithmetic operation on the provided input tensors. |
| can_cast | Determines if a type conversion is allowed under PyTorch casting rules described in the type promotion documentation. |
| promote_types | Returns the torch.dtype with the smallest size and scalar kind that is not smaller nor of lower kind than either type1 or type2. |
| use_deterministic_algorithms | Sets whether PyTorch operations must use “deterministic” algorithms. |
| are_deterministic_algorithms_enabled | Returns True if the global deterministic flag is turned on. |
| is_deterministic_algorithms_warn_only_enabled | Returns True if the global deterministic flag is set to warn only. |
| set_deterministic_debug_mode | Sets the debug mode for deterministic operations. |
| get_deterministic_debug_mode | Returns the current value of the debug mode for deterministic operations. |
| set_float32_matmul_precision | Sets the internal precision of float32 matrix multiplications. |
| get_float32_matmul_precision | Returns the current value of float32 matrix multiplication precision. |
| set_warn_always | When this flag is False (default) then some PyTorch warnings may only appear once per process. |
| is_warn_always_enabled | Returns True if the global warn_always flag is turned on. |
| _assert | A wrapper around Python’s assert which is symbolically traceable. |
PyTorch 的算子大多为异步惰性求值，尤其是 GPU 版本。
算子只定义了入口，真正的实现依赖于不同的后端，常用的有 cpu、cuda、mkldnn、npu 等等。
知道这些的意义是，不用管这些，爱怎么编程就怎么编就可以了，PyTorch 引擎会帮你搞定一切的。
