Transformers v4.12.0 compatible by JulianneKnott · Pull Request #107 · microsoft/fastseq

Julianne Knott (JulianneKnott) · 2021-11-22T18:00:41Z

Updating Fastseq for compatibility with huggingface transformers v4.12.0.
Benchmarks (samples/sec):

Bart 13.0 (2.9x)
DistilBart 19.1 (3.5x)
T5 29.2 (2.0x)
GPT2 20.3 (5.1x)
ProphetNet TBD

Fei Hu (feihugis)

Julianne Knott (@JulianneKnott) Thanks for updating HF models! I did a pass and left some minor comments. Did you check if all the changed code have been used during the inference?

Jiusheng Chen (JiushengChen) · 2021-11-30T19:15:14Z

- #https://github.com/huggingface/transformers.git \
+ export BASELINE_REPO=$CACHE_DIR/transformers_v4.12.0
 git_clone_if_not_in_cache \
-	https://github.com/JiushengChen/transformers.git \


Some context about this forked repo, it is to add param "no_repeat_ngram_size", see
JiushengChen/Transformers@db97043

To clarify, this is no longer needed since fastseq uses it's own run_eval_hf.py for the baseline?

Fei Hu (feihugis) · 2021-12-13T18:20:27Z

+ is_greedy_gen_mode = (num_beams == 1) and (num_beam_groups == 1) and do_sample is False
+ is_sample_gen_mode = (num_beams == 1) and (num_beam_groups == 1) and do_sample is True
+ is_beam_gen_mode = (num_beams > 1) and (num_beam_groups == 1) and do_sample is False
+ is_beam_sample_gen_mode = (num_beams > 1) and (num_beam_groups == 1) and do_sample is True
+ is_group_beam_gen_mode = (num_beams > 1) and (num_beam_groups > 1)


Are our changes compatible with all these generation modes?

The beam search updates are only applied for is_beam_gen_mode. The model-specific updates (ie attention) are applied in all cases.

Does it also work with is_group_beam_gen_mode?

no is_group_beam_gen_mode does not use the updates. Should that be added?

If the optimizations can also work for group_beam_gen_mode, we can add it. We can support it in the another PR later if it will take some time to support it.

I think it will take some time to make it work for both, so I think it's best if group_beam_gen_mode goes in another pr

Sounds good.

Fei Hu (feihugis)

Thanks Julianne Knott (@JulianneKnott) for the revisions! The PR looks good to me!

Julianne Knott (JulianneKnott) added 15 commits October 28, 2021 17:42

update transformers version number
12b9985

update transformer version number (4.12.0)
e2b7a1d

bart and gpt2 updates (no el attn, old cli)
2566867

t5 updates (no el attn, old cli)
434185c

debug
2a8f779

updated (existing fastseq optimization)
67bf684

bart, gpt2, t5 benchmarks updated
6f8e451

add prophetnet
c3fc882

add distilbart benchmark
c6b98b6

import prophetnet optimizer
f00c752

simplify prophetnet test
9c32b81

update benchmarks checks
940a78d

prophetnet benchmarks
39b33ba

cleanup
f4ce9c2

cleanup
2ac5317

Fei Hu (feihugis) reviewed Nov 22, 2021
View reviewed changes

Julianne Knott (JulianneKnott) added 7 commits November 23, 2021 17:57

cleanup
08cc31a

azure pipeline updates
bd58d06

sentencepiece update (docker + azure pipeline)
d9466e4

pipelines update torch=1.10.0
cc4494c

fix prophetnet fs expected output + cleanup
c3c98cb

benchmarks, examples, + readmes
e4354c3

fixed t5 ngram repeat block
9fb69a7

Jiusheng Chen (JiushengChen) reviewed Nov 30, 2021
View reviewed changes

Julianne Knott (JulianneKnott) added 6 commits December 2, 2021 01:03

fix gpt2, t5, prophetnet unit hf tests
298ce3a

fixed tests broken by fastseq update
b6e899e

unit test cleanup (w/ hf docker)
6faf3fb

update pipeline image
e549380

update pipeline
e1e76d5

pipeline debug
8fb5575

Julianne Knott (JulianneKnott) added 17 commits December 9, 2021 02:33

debug
f9e8c16

debug
c464dee

debug
addd2a9

debug
c61bedb

pipeline debug
645887b

debug pipeline
cc34fc7

debug pipeline
8490d64

debug pipeline
1e43d3f

fastseq tests working, run fairseq tests
816cab9

debug venv
124f768

debug
7aab0d5

debug
f28672d

fairseq tests working, run with fastseq tests
7d83023

try running all tests
1633ef3

typo fix
49beb3d

fixed last 4 failing tests
980efa6

cleanup
0ad879e

Fei Hu (feihugis) reviewed Dec 13, 2021
View reviewed changes

Julianne Knott (JulianneKnott) added 2 commits December 13, 2021 18:39

compatibility up to 4.12.5
943f198

compatibility
5233d86

Fei Hu (feihugis) reviewed Dec 13, 2021
View reviewed changes

Julianne Knott (JulianneKnott) added 4 commits December 13, 2021 18:51

cleanup
31a49ff

fix bart benchmarks
8ad78b3

cleanup
d501760

cleanup
61fb12a

Fei Hu (feihugis) approved these changes Dec 13, 2021
View reviewed changes

prophetnet benchmarks
c66ec06

Fei Hu (feihugis) reviewed Dec 14, 2021
View reviewed changes

Comment threadexamples/prophetnet/README.md Outdated

typo fixed
3513f0f

Julianne Knott (JulianneKnott) merged commit df98314 into mainDec 14, 2021

Uh oh!

Conversation

Julianne Knott (JulianneKnott) commented Nov 22, 2021

Uh oh!

Fei Hu (feihugis) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Fei Hu (feihugis) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants