Code

LightSeq QAT#307

Code

Merged

Taka152 merged 132 commits intomasterbytedance/lightseq:masterfrom ls-qatbytedance/lightseq:ls-qatCopy head branch name to clipboardAug 31, 2022

+33,917-922

Commits

Commits on Apr 13, 2022

• ls embedding support qat

  

  Ying Xiong

  committed
  13ac633

Commits on May 5, 2022

• [WIP]ls transformer qat

  

  Ying Xiong

  committed
  ace049a

Commits on May 6, 2022

• fix fairseq transformer cli shape bug of output projection

  godweiyang

  committed
  23041c2

Commits on May 7, 2022

• ln_bw_i8 test passed!

  godweiyang

  committed
  5c6ad24

• test with_mean of ln_i8

  godweiyang

  committed
  576de44

• update to the latest version of master, fix conflict

  godweiyang

  committed
  178b774

Commits on May 9, 2022

• ls encoder attn add qat

  

  Ying Xiong

  committed
  2c6c30b

Commits on May 10, 2022

• dropout_relu_bias_i8 passed!

  godweiyang

  committed
  580775c

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  dd1229b

Commits on May 16, 2022

• dropout_gelu_bias unit test passed!

  godweiyang

  committed
  75eb4f4

• dropout_relu_bias_bwd_i8 passed!

  godweiyang

  committed
  e4b8e9c

• Merge branch 'ls-qat-wy' into ls-qat

  godweiyang

  committed
  0b0c890

Commits on May 17, 2022

• dropout_gelu_bias_bwd_i8 unit test passed!

  godweiyang

  committed
  6fab849

• format

  godweiyang

  committed
  b5ab256

• dropout_gelu_bias_bwd_i8 unit test passed!

  godweiyang

  committed
  f2f8401

• format

  godweiyang

  committed
  59e5ebf

Commits on May 18, 2022

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  d39f15c

• polish unit test

  godweiyang

  committed
  ffda943

Commits on May 23, 2022

• [WIP] ls encoder qat test

  

  Ying Xiong

  committed
  5370059

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  

  Ying Xiong

  committed
  e66708c

• quant_bias_add_transform_20314, quant_transform4d_0213 unit test passed!

  godweiyang

  committed
  20231b6

Commits on May 26, 2022

• fix unit test bug

  godweiyang

  committed
  991ca74

Commits on May 30, 2022

• [WIP] ls encoder qat unit test

  

  Ying Xiong

  committed
  a8af545

• fix bug

  godweiyang

  committed
  e03da8a

• set default module to disable quant, fix bugs in examples

  godweiyang

  committed
  238f717

Commits on Jun 1, 2022

• fix encoder bug

  

  Ying Xiong

  committed
  1e035a8

Commits on Jun 6, 2022

• encoder qat test pass

  

  Ying Xiong

  committed
  b6ec156

Commits on Jun 8, 2022

• decoder qat forward test pass

  

  Ying Xiong

  committed
  f934ca0

Commits on Jun 9, 2022

• fix bug in encoder bw

  godweiyang

  committed
  3d15555

• fix conflict

  godweiyang

  committed
  a98e6b5

Commits on Jun 10, 2022

• fix bug of cmax grad

  godweiyang

  committed
  7d06ffc

• fix bug of act mask

  godweiyang

  committed
  0a8fc46

• fix bug in tensor quantizer

  godweiyang

  committed
  89208b1

Commits on Jun 13, 2022

• fix cmax grad bug

  godweiyang

  committed
  09a3d10

Commits on Jun 14, 2022

• [WIP] decoder support qat

  

  Ying Xiong

  committed
  07e376d

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  

  Ying Xiong

  committed
  c57b398

Commits on Jun 20, 2022

• ls decoder qat pass

  

  Ying Xiong

  committed
  a380b72

Commits on Jun 22, 2022

• ls encoder qat pass

  

  Ying Xiong

  committed
  09f6bf5

Commits on Jul 5, 2022

• add unit test for quant bert encoder

  godweiyang

  committed
  4e2b257

Commits on Jul 11, 2022

• fix memory bug

  godweiyang

  committed
  956f681

Commits on Jul 12, 2022

• fix cmax grad bug in huggingface

  godweiyang

  committed
  1c9ddb4

Commits on Jul 14, 2022

• quant bert enc fw&bw test passed!

  godweiyang

  committed
  db9d160

• fix hf cmax export bug

  godweiyang

  committed
  e6338f7

• fix fairseq out_proj bug

  godweiyang

  committed
  d508f62

• fix fairseq shell bug

  godweiyang

  committed
  c641567

Commits on Jul 15, 2022

• fix decoder mem bug

  godweiyang

  committed
  60a368e

• modify initial lr of fairseq quant training

  godweiyang

  committed
  b18995f

Commits on Jul 18, 2022

• decoupled qat code

  

  Ying Xiong

  committed
  04d1291

• modify huggingface training scripts

  godweiyang

  committed
  972a54d

• decoupled qat code

  godweiyang

  committed
  7ea40bd

• add cmax grad

  

  Ying Xiong

  committed
  1b117f0

• delete enc_kv output quant

  godweiyang

  committed
  1859341

Commits on Jul 20, 2022

• modify ffn2gemm quant like inference

  godweiyang

  committed
  7d4a8cd

• fuse dequantize

  

  xiongying.taka

  committed
  b569eb7

• fix post ln mem bug

  godweiyang

  committed
  94ffe31

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  d596b29

Commits on Jul 21, 2022

• add decoder self attn qkv cache quant

  godweiyang

  committed
  eb6c59a

• export quant model (stage 1)

  godweiyang

  committed
  9cd64f5

Commits on Jul 22, 2022

• export quant model (stage 2)

  godweiyang

  committed
  b5ab18c

• export quant model (stage 3)

  godweiyang

  committed
  42607a1

Commits on Jul 26, 2022

• support vit quant train

  godweiyang

  committed
  8358e48

• add gradient clip

  

  xiongying.taka

  committed
  5d40f93

• fix hf export bug

  godweiyang

  committed
  725f112

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  28b2b76

• fix quant gpt bug

  godweiyang

  committed
  075af1a

• support quant gpt training

  godweiyang

  committed
  b622b0d

Commits on Jul 27, 2022

• modify huggingface training scripts

  godweiyang

  committed
  d3d3566

• support ls bert, gpt export

  godweiyang

  committed
  baba9d7

Commits on Jul 28, 2022

• support custom quant transformer export

  godweiyang

  committed
  3bbfac3

• optimizer ffn fake quant and dcmax

  

  xiongying.taka

  committed
  483c252

• support quant gpt export

  godweiyang

  committed
  0983fe0

Commits on Jul 29, 2022

• support quant vit export

  godweiyang

  committed
  bfa3b76

• add quant linear layer

  

  xiongying.taka

  committed
  0c21296

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  

  xiongying.taka

  committed
  89a92e7

• fix quant linear layer bug

  

  xiongying.taka

  committed
  e683775

Commits on Aug 1, 2022

• support quant vit infer

  godweiyang

  committed
  c180d2c

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  e8ad835

Commits on Aug 2, 2022

• speedup cublass igemm on A100 (by huxingwu)

  godweiyang

  committed
  931c0b0

• optimize ls_quant_dropout_act_bias_bwd_kernel

  

  xiongying.taka

  committed
  0fe0862

Commits on Aug 3, 2022

• polish training gemm algo code

  godweiyang

  committed
  ee03974

Commits on Aug 5, 2022

• support gemm best algo search on different GPUs and shapes

  godweiyang

  committed
  56211cb

• search in the range (min_bsz, 512, 1) and (512, max_bsz, 32)

  godweiyang

  committed
  2e99f37

• add configs_sm75/h512_i2048_b1-10016.json

  godweiyang

  committed
  e2439e1

• support col32 igemm

  

  xiongying.taka

  committed
  156bb6e

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  

  xiongying.taka

  committed
  477d3e7

• add configs_sm75/h768_i3072_b1-10016.json

  godweiyang

  committed
  01dccf8

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  6f13034

• add configs_sm80/h512_i2048_b1-10016.json

  godweiyang

  committed
  4fbbb0f

• add configs_sm75/h1024_i4096_b1-10016.json

  godweiyang

  committed
  8305cd3

• add configs_sm80/h768_i3072_b1-10016.json

  godweiyang

  committed
  3f02450

• fix syntax error

  godweiyang

  committed
  c0c9d81

• configs_sm80/h1024_i4096_b1-10016.json

  godweiyang

  committed
  1b23508

Commits on Aug 8, 2022

• modify gemm test config format

  godweiyang

  committed
  d521929

• merge all the configs to one

  godweiyang

  committed
  3868f59

• support search all shapes which are not in the config

  godweiyang

  committed
  33702dd

• polish the merged config

  godweiyang

  committed
  a9104d0

Commits on Aug 9, 2022

• add cublas_algo_map cpp code

  godweiyang

  committed
  430e4e4

• move get_sm func to lightseq kernels

  godweiyang

  committed
  aa7d4a1

• move gemm_test to lightseq ops

  godweiyang

  committed
  cf21734

• modify default config dir, fix algo_map bug

  godweiyang

  committed
  e582959

Commits on Aug 10, 2022

• fix col32 bug

  

  xiongying.taka

  committed
  d91024a

Commits on Aug 11, 2022

• col major igemm become default

  

  xiongying.taka

  committed
  690801e

• fix dcax kernel bug

  

  xiongying.taka

  committed
  2877db5

Commits on Aug 12, 2022

• loosen cuda 11.6 requirement

  

  xiongying.taka

  committed
  649a037

• add vit cpp example

  godweiyang

  committed
  fc6a035

• Merge branch 'ls-qat' of https://github.com/bytedance/lightseq into ls-qat

  godweiyang

  committed
  0a76d2e

Commits on Aug 15, 2022

• fix bug from col32 gemm and a100 tuned col gemm

  

  xiongying.taka

  committed
  cd9c85c

Commits on Aug 16, 2022

• support training encoder qkv_linear auto-tune gemm (in comment)

  godweiyang

  committed
  7eac0ed

• add required header file

  godweiyang

  committed
  7a1bd0c

• dynamic use col32 or col4 in different GPUs

  godweiyang

  committed
  1904c74

• fix multidefinition bug

  godweiyang

  committed
  220c2e8

Commits on Aug 17, 2022

• fix weight transform col32 bug

  godweiyang

  committed
  964ff11

Commits on Aug 18, 2022

• add best algo for inference gemm (in comments)

  godweiyang

  committed
  50f6512

Commits on Aug 19, 2022

• support easy benchmark for gpt and transformer

  

  xiongying.taka

  committed
  78857bb

Commits on Aug 22, 2022

• support benmark huggingface

  

  xiongying.taka

  committed
  36b6607

Commits on Aug 23, 2022

• fix embedding clip_max bug

  

  duanrenchong

  committed
  efa5c71

• ls quant linear support more shape

  

  xiongying.taka

  committed
  3e63d41

Commits on Aug 24, 2022

• fix quant linear bug

  

  xiongying.taka

  committed
  12d58ed

• fix quant linear bug

  

  xiongying.taka

  committed
  c8e5b89

• update pad function for older torch

  

  xiongying.taka

  committed
  41e5cec

• fix quant linear bug

  

  xiongying.taka

  committed
  33cc7b0

• remove redundant code

  

  xiongying.taka

  committed
  123874a

Commits on Aug 26, 2022

• Merge branch 'master' into ls-qat

  

  xiongying.taka

  committed
  8dac833

• fix export bug

  

  xiongying.taka

  committed
  332e22e

• Merge remote-tracking branch 'origin/master' into ls-qat

  

  xiongying.taka

  committed
  a8df5d6

• fix format

  

  xiongying.taka

  committed
  11af3e7

Commits on Aug 30, 2022

• fix conflicts

  godweiyang

  committed
  389106d

• fix custom train&infer bug

  godweiyang

  committed
  a62cad9

• fix quant infer size overflow

  godweiyang

  committed
  541ec0f

• fix ls gpt export bug (extra_decode_length)

  godweiyang

  committed
  91d7c1a

• fix hf bart cmax init and state

  

  duanrenchong

  committed
  a42a98c

• fix max-batch-tokens bug of bart predict

  godweiyang

  committed
  2e8fe5a