CANN/asc-devkit Cube指令理论性能汇总

Cube指令理论性能汇总【免费下载链接】asc-devkit本项目是CANN 推出的昇腾AI处理器专用的算子程序开发语言原生支持C和C标准规范主要由类库和语言扩展层构成提供多层级API满足多维场景算子开发诉求。项目地址: https://gitcode.com/cann/asc-devkit本节汇总介绍了主要的Cube指令的理论性能以下内容针对如下型号生效Ascend 950PR/Ascend 950DTAtlas A3 训练系列产品/Atlas A3 推理系列产品Atlas A2 训练系列产品/Atlas A2 推理系列产品Mmad计算类指令理论性能汇总以下为Mmad计算类指令理论性能计算公式在不同shape大小的情况下unitFlag的开关对性能几乎无影响。$$ \begin{gathered} {ceil_m} \left\lceil \frac{m}{16} \right\rceil \times 16 \[12pt] {ceil_n} \left\lceil \frac{n}{16} \right\rceil \times 16 \[12pt] {ceil_k} \left\lceil \frac{k_0}{16} \right\rceil \times k_0 \[16pt] \text{cube利用率} \frac{ (m \times n \times k) / ({cube_m} \times {cube_n} \times {cube_k}) } { \Delta t ({ceil_m} \times {ceil_n} \times {ceil_k}) / ({cube_m} \times {cube_n} \times {cube_k}) } \end{gathered} $$关键变量及常量说明$m, n, k$mmad入参实际计算的大小。$ceil_m, ceil_n, ceil_k$$m, n, k$ 根据分型大小向上对齐后的值。$cube_m, cube_n, cube_k$硬件真实并行度。$k_0$L0 Buffer上最小分型K方向大小。$\Delta t$头开销cycle数。表1Mmad计算类指令理论性能计算公式中并行度和k0的取值NPU架构版本2201接口左矩阵A右矩阵Bcubemcubencubekk0Mmadint4b_tint4b_t16166464Mmadint8_tint8_t16163232Mmadhalfhalf16161616Mmadbfloat16_tbfloat16_t16161616Mmadfloatfloat161648Mmad开启HF32floatfloat161688MmadWithSparseint8_tint8_t16163232表2Mmad计算类指令理论性能计算公式中并行度和k0的取值NPU架构版本3510接口左矩阵A右矩阵Bcubemcubencubekk0Mmadint8_tint8_t16163232Mmadfp8_e4m3fn_tfp8_e4m3fn_t16163232Mmadfp8_e4m3fn_tfp8_e5m2_t16163232Mmadfp8_e5m2_tfp8_e4m3fn_t16163232Mmadfp8_e5m2_tfp8_e5m2_t16163232Mmadhifloat8_thifloat8_t16163232Mmadhalfhalf16161616Mmadbfloat16_tbfloat16_t16161616Mmadfloatfloat161618Mmad开启HF32floatfloat161688MmadMxfp4x2_e1m2_tfp4x2_e1m2_t16166464MmadMxfp4x2_e2m1_tfp4x2_e1m2_t16166464MmadMxfp4x2_e1m2_tfp4x2_e2m1_t16166464MmadMxfp4x2_e2m1_tfp4x2_e2m1_t16166464MmadMxfp8_e4m3fn_tfp8_e4m3fn_t16163232MmadMxfp8_e4m3fn_tfp8_e5m2_t16163232MmadMxfp8_e5m2_tfp8_e4m3fn_t16163232MmadMxfp8_e5m2_tfp8_e5m2_t16163232矩阵计算搬入类指令理论性能汇总以下内容为矩阵计算搬入类指令在L1 Buffer-L0A Buffer、L1 Buffer-L0B Buffer、L1 Buffer-L0A_MX Buffer、L1 Buffer-L0B_MX Buffer、L1 Buffer-BiasTable Buffer、L1 Buffer-Fixpipe Buffer多条数据搬运通路下的理论性能汇总说明。数据搬运指令占用cycle数的理论计算公式$$ {cycles} {data_size} / {bandwidth} \Delta T $$关键变量及常量说明$data_size$搬运的数据总量单位为Byte。$bandwidth$搬运的带宽每个cycle能够搬运的数据量大小单位为Byte/cycle。$\Delta T$搬运过程中的带宽延迟等额外开销不同搬运指令的参考值不相同。注意下列表格中展示的理论性能为搬运带宽$bandwidth$未考虑额外开销$\Delta T$。表3矩阵计算搬入类指令理论性能说明NPU架构版本2201接口搬运带宽单位Byte/cycleLoadData2D矩阵搬运L1 Buffer-L0A Buffer256LoadData2D矩阵搬运L1 Buffer-L0B Buffer128LoadDataWithTransposeL1 Buffer-L0A Buffer256LoadDataWithTransposeL1 Buffer-L0B Buffer128LoadDataWithSparseL1 Buffer-L0B Buffer128DataCopyL1 Buffer-BiasTable Buffer32DataCopyL1 Buffer-Fixpipe Buffer32LoadData卷积数据搬运指令的理论性能与参数相关相关参数说明见LoadData3DParamsV2结构体内参数说明理论性能说明见下表表4LoadData卷积数据搬运指令理论性能说明NPU架构版本2201strideW的取值卷积核在H方向滑动的行数nL1 Buffer-L0A Buffer带宽单位Byte/cycleL1 Buffer-L0B Buffer带宽单位Byte/cycle112561281大于1$\left[\frac{512}{n2}\right]$最小不会低于32$\left[\frac{512}{n4}\right]$最小不会低于$\left[\frac{512}{162}\right]$2、4、81$\left[\frac{512}{(strideW \times 2)}\right]$最小不会低于32$\left[\frac{512}{(strideW \times 2 2)}\right]$最小不会低于$\left[\frac{512}{162}\right]$2、4、8大于1$\left[\frac{512}{(strideW \times 2 n)}\right]$最小不会低于32$\left[\frac{512}{(strideW \times 2 2 n)}\right]$最小不会低于$\left[\frac{512}{162}\right]$大于1且不等于2、4、8任意值32$\left[\frac{512}{162}\right]$表5矩阵计算搬入类指令理论性能说明NPU架构版本3510接口搬运带宽单位Byte/cycleLoadData2D矩阵搬运V2L1 Buffer-L0A Buffer256LoadData2D矩阵搬运V2L1 Buffer-L0B Buffer256LoadDataMX矩阵搬运L1 Buffer-L0A Buffer256LoadDataMX矩阵搬运L1 Buffer-L0B Buffer256LoadDataMX矩阵搬运L1 Buffer-L0A_MX Buffer32LoadDataMX矩阵搬运L1 Buffer-L0B_MX Buffer32LoadDataWithTransposeL1 Buffer-L0A Buffer256LoadDataWithTransposeL1 Buffer-L0B Buffer128DataCopyL1 Buffer-BiasTable Buffer32DataCopyL1 Buffer-Fixpipe Buffer32针对Ascend 950PR/Ascend 950DTLoadData2D矩阵搬运接口仅为兼容实现内部使用了LoadData2D矩阵搬运V2接口实现相关理论性能可参考LoadData2D矩阵搬运V2接口但需要注意该兼容实现会造成性能损失。由于LoadDataMX矩阵搬运接口内部实现涉及两条数据通路的搬运其占用cycle数的理论计算公式有所不同具体公式如下$$ {cycles} {data_size} / {bandwidth} {scale_data_size} / {bandwidth_mx} \Delta T $$关键变量及常量说明$data_size$搬运左/右矩阵的数据总量单位为Byte。$scale_data_size$搬运scale矩阵的数据总量单位为Byte。$bandwidth$搬运左/右矩阵的带宽单位为Byte/cycle。$bandwidth_mx$搬运scale矩阵的带宽单位为Byte/cycle。$\Delta T$搬运过程中的带宽延迟等额外开销。针对Ascend 950PR/Ascend 950DTLoadData卷积数据搬运和LoadDataWithStride指令的理论性能与参数相关相关参数说明见LoadData3DParamsV2结构体内参数说明。此外LoadDataWithStride接口相比于LoadData卷积数据搬运接口内部不包含针对其他芯片版本的兼容性实现减少了兼容造成的额外开销性能表现有所优化。两个接口的理论性能说明见下表表6LoadData卷积数据搬运和LoadDataWithStride理论性能说明NPU架构版本3510strideW的取值卷积核在H方向滑动的行数nL1 Buffer-L0A Buffer带宽单位Byte/cycleL1 Buffer-L0B Buffer带宽单位Byte/cycle112562561大于1$\left[\frac{512}{n2}\right]$最小不会低于32$\left[\frac{512}{n2}\right]$最小不会低于322、4、81$\left[\frac{512}{(strideW \times 2)}\right]$最小不会低于32$\left[\frac{512}{(strideW \times 2)}\right]$最小不会低于322、4、8大于1$\left[\frac{512}{(strideW \times 2 n)}\right]$最小不会低于32$\left[\frac{512}{(strideW \times 2 n)}\right]$最小不会低于32大于1且不等于2、4、8任意值3232矩阵计算搬出类指令理论性能汇总以下内容为矩阵计算搬出类指令在L0C Buffer-L1 Buffer和L0C Buffer-Unified Buffer两条数据搬运通路下的理论性能汇总说明。数据搬运指令占用cycle数的理论计算公式$$ {cycles} {data_size} / {bandwidth} \Delta T $$关键变量及常量说明$data_size$搬运的数据总量单位为Byte。$bandwidth$搬运的带宽每个cycle能够搬运的数据量大小单位为Byte/cycle。$\Delta T$搬运过程中的带宽延迟等额外开销不同搬运指令的参考值不相同。注意下列表格中展示的理论性能为搬运带宽$bandwidth$未考虑额外开销$\Delta T$。矩阵计算搬出类指令及其随路功能的详细介绍可参考L0C到GM数据搬运DataCopy、L0C到GM数据搬运Fixpipe以及相同目录下的其他通路对应资料。以下为理论性能说明列表表7矩阵计算搬出类指令理论性能说明NPU架构版本2201数据通路L0C Buffer上的数据类型量化模式格式转换模式目的内存上的数据类型并行度单位elements/cycle搬运带宽单位Byte/cycleL0C Buffer-L1 BufferfloatF322F16NZ2NZhalf64128L0C Buffer-L1 BufferfloatF322BF16NZ2NZbfloat16_t64128L0C Buffer-L1 Bufferfloat(V)QF322B8_PRENZ2NZ Channel Mergeint8_t/uint8_t6464L0C Buffer-L1 Bufferint32_t(V)DEQF16NZ2NZhalf64128L0C Buffer-L1 Bufferint32_t(V)REQ8NZ2NZ Channel Mergeint8_t/uint8_t6464表8矩阵计算搬出类指令理论性能说明NPU架构版本3510数据通路L0C Buffer上的数据类型量化模式格式转换模式目的内存上的数据类型并行度单位elements/cycle搬运带宽单位Byte/cycleL0C Buffer-L1 BufferfloatNoQuant/(V)QF322F32_PRENZ2NZfloat32128L0C Buffer-L1 BufferfloatNoQuant/(V)QF322F32_PRENZ2NZ Channel Splitfloat32128L0C Buffer-L1 BufferfloatNoQuant/(V)QF322F32_PRENZ2NDfloat32128L0C Buffer-L1 BufferfloatNoQuant/(V)QF322F32_PRENZ2DN srcNzC0Stride 1float32128L0C Buffer-L1 BufferfloatNoQuant/(V)QF322F32_PRENZ2DN srcNzC0Stride 1float832L0C Buffer-L1 BufferfloatF322F16/(V)QF322F16_PRENZ2NZhalf64128L0C Buffer-L1 BufferfloatF322F16/(V)QF322F16_PRENZ2NDhalf64128L0C Buffer-L1 BufferfloatF322F16/(V)QF322F16_PRENZ2DN srcNzC0Stride 1half64128L0C Buffer-L1 BufferfloatF322F16/(V)QF322F16_PRENZ2DN srcNzC0Stride 1half1632L0C Buffer-L1 BufferfloatF322BF16/(V)QF322BF16_PRENZ2NZbfloat16_t64128L0C Buffer-L1 BufferfloatF322BF16/(V)QF322BF16_PRENZ2NDbfloat16_t64128L0C Buffer-L1 BufferfloatF322BF16/(V)QF322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t64128L0C Buffer-L1 BufferfloatF322BF16/(V)QF322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t1632L0C Buffer-L1 Bufferfloat(V)QF322FP8_PRENZ2NZ Channel Mergefp8_e4m3fn_t6464L0C Buffer-L1 Bufferfloat(V)QF322FP8_PRENZ2NDfp8_e4m3fn_t6464L0C Buffer-L1 Bufferfloat(V)QF322FP8_PRENZ2DN srcNzC0Stride 1fp8_e4m3fn_t6464L0C Buffer-L1 Bufferfloat(V)QF322FP8_PRENZ2DN srcNzC0Stride 1fp8_e4m3fn_t1616L0C Buffer-L1 Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2NZ Channel Mergehifloat8_t6464L0C Buffer-L1 Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2NDhifloat8_t6464L0C Buffer-L1 Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2DN srcNzC0Stride 1hifloat8_t6464L0C Buffer-L1 Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2DN srcNzC0Stride 1hifloat8_t1616L0C Buffer-L1 Bufferfloat(V)QF322B8_PRENZ2NZ Channel Mergeint8_t/uint8_t6464L0C Buffer-L1 Bufferfloat(V)QF322B8_PRENZ2NDint8_t/uint8_t6464L0C Buffer-L1 Bufferfloat(V)QF322B8_PRENZ2DN srcNzC0Stride 1int8_t/uint8_t6464L0C Buffer-L1 Bufferfloat(V)QF322B8_PRENZ2DN srcNzC0Stride 1int8_t/uint8_t1616L0C Buffer-L1 Bufferint32_t(V)DEQF16NZ2NZhalf64128L0C Buffer-L1 Bufferint32_t(V)DEQF16NZ2NDhalf64128L0C Buffer-L1 Bufferint32_t(V)DEQF16NZ2DN srcNzC0Stride 1half64128L0C Buffer-L1 Bufferint32_t(V)DEQF16NZ2DN srcNzC0Stride 1half1632L0C Buffer-L1 Bufferint32_t(V)QS322BF16_PRENZ2NZbfloat16_t64128L0C Buffer-L1 Bufferint32_t(V)QS322BF16_PRENZ2NDbfloat16_t64128L0C Buffer-L1 Bufferint32_t(V)QS322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t64128L0C Buffer-L1 Bufferint32_t(V)QS322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t1632L0C Buffer-L1 Bufferint32_t(V)REQ8NZ2NZ Channel Mergeint8_t/uint8_t6464L0C Buffer-L1 Bufferint32_t(V)REQ8NZ2NDint8_t/uint8_t6464L0C Buffer-L1 Bufferint32_t(V)REQ8NZ2DN srcNzC0Stride 1int8_t/uint8_t6464L0C Buffer-L1 Bufferint32_t(V)REQ8NZ2DN srcNzC0Stride 1int8_t/uint8_t1616L0C Buffer-Unified Buffer开启双目标模式floatNoQuantNZ2NZfloat32 32128 128L0C Buffer-Unified Buffer开启双目标模式floatNoQuantNZ2NDfloat32 32128 128L0C Buffer-Unified BufferfloatNoQuant/(V)QF322F32_PRENZ2NZfloat32128L0C Buffer-Unified BufferfloatNoQuant/(V)QF322F32_PRENZ2NZ Channel Splitfloat32128L0C Buffer-Unified BufferfloatNoQuant/(V)QF322F32_PRENZ2NDfloat32128L0C Buffer-Unified BufferfloatNoQuant/(V)QF322F32_PRENZ2DN srcNzC0Stride 1float32128L0C Buffer-Unified BufferfloatNoQuant/(V)QF322F32_PRENZ2DN srcNzC0Stride 1float832L0C Buffer-Unified BufferfloatF322F16/(V)QF322F16_PRENZ2NZhalf64128L0C Buffer-Unified BufferfloatF322F16/(V)QF322F16_PRENZ2NDhalf64128L0C Buffer-Unified BufferfloatF322F16/(V)QF322F16_PRENZ2DN srcNzC0Stride 1half64128L0C Buffer-Unified BufferfloatF322F16/(V)QF322F16_PRENZ2DN srcNzC0Stride 1half1632L0C Buffer-Unified BufferfloatF322BF16/(V)QF322BF16_PRENZ2NZbfloat16_t64128L0C Buffer-Unified BufferfloatF322BF16/(V)QF322BF16_PRENZ2NDbfloat16_t64128L0C Buffer-Unified BufferfloatF322BF16/(V)QF322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t64128L0C Buffer-Unified BufferfloatF322BF16/(V)QF322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t1632L0C Buffer-Unified Bufferfloat(V)QF322FP8_PRENZ2NZ Channel Mergefp8_e4m3fn_t6464L0C Buffer-Unified Bufferfloat(V)QF322FP8_PRENZ2NDfp8_e4m3fn_t6464L0C Buffer-Unified Bufferfloat(V)QF322FP8_PRENZ2DN srcNzC0Stride 1fp8_e4m3fn_t6464L0C Buffer-Unified Bufferfloat(V)QF322FP8_PRENZ2DN srcNzC0Stride 1fp8_e4m3fn_t1616L0C Buffer-Unified Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2NZ Channel Mergehifloat8_t6464L0C Buffer-Unified Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2NDhifloat8_t6464L0C Buffer-Unified Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2DN srcNzC0Stride 1hifloat8_t6464L0C Buffer-Unified Bufferfloat(V)QF322HIF8_PRE/(V)QF322HIF8_PRE_HYBRIDNZ2DN srcNzC0Stride 1hifloat8_t1616L0C Buffer-Unified Bufferfloat(V)QF322B8_PRENZ2NZ Channel Mergeint8_t/uint8_t6464L0C Buffer-Unified Bufferfloat(V)QF322B8_PRENZ2NDint8_t/uint8_t6464L0C Buffer-Unified Bufferfloat(V)QF322B8_PRENZ2DN srcNzC0Stride 1int8_t/uint8_t6464L0C Buffer-Unified Bufferfloat(V)QF322B8_PRENZ2DN srcNzC0Stride 1int8_t/uint8_t1616L0C Buffer-Unified Bufferint32_t(V)DEQF16NZ2NZhalf64128L0C Buffer-Unified Bufferint32_t(V)DEQF16NZ2NDhalf64128L0C Buffer-Unified Bufferint32_t(V)DEQF16NZ2DN srcNzC0Stride 1half64128L0C Buffer-Unified Bufferint32_t(V)DEQF16NZ2DN srcNzC0Stride 1half1632L0C Buffer-Unified Bufferint32_t(V)QS322BF16_PRENZ2NZbfloat16_t64128L0C Buffer-Unified Bufferint32_t(V)QS322BF16_PRENZ2NDbfloat16_t64128L0C Buffer-Unified Bufferint32_t(V)QS322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t64128L0C Buffer-Unified Bufferint32_t(V)QS322BF16_PRENZ2DN srcNzC0Stride 1bfloat16_t1632L0C Buffer-Unified Bufferint32_t(V)REQ8NZ2NZ Channel Mergeint8_t/uint8_t6464L0C Buffer-Unified Bufferint32_t(V)REQ8NZ2NDint8_t/uint8_t6464L0C Buffer-Unified Bufferint32_t(V)REQ8NZ2DN srcNzC0Stride 1int8_t/uint8_t6464L0C Buffer-Unified Bufferint32_t(V)REQ8NZ2DN srcNzC0Stride 1int8_t/uint8_t1616注srcNzC0Stride为启用NZ2DN功能需配置的参数之一参数详细介绍请参考NZ2DN关键特性说明。【免费下载链接】asc-devkit本项目是CANN 推出的昇腾AI处理器专用的算子程序开发语言原生支持C和C标准规范主要由类库和语言扩展层构成提供多层级API满足多维场景算子开发诉求。项目地址: https://gitcode.com/cann/asc-devkit创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考