Text Summarization- Notes

The Evolution:

1. Huggingface Transformers

2. BERT

3. GPT

4. BART- BERT+GPT

5. T5

6. GPT-3- can't be used in production

7. Electra- uses an approach similar to GANs and RL- best for any generic NLP task

8. Pegasus- Electra tuned specifically for Text Summarization

References:

https://towardsdatascience.com/comparing-text-summarization-techniques-d1e2e465584e

https://blog.floydhub.com/gentle-introduction-to-text-summarization-in-machine-learning/

https://www.quora.com/What-are-the-best-algorithms-for-text-summarization-either-supervised-or-unsupervised

https://www.machinelearningplus.com/nlp/text-summarization-approaches-nlp-example/

https://www.analyticsvidhya.com/blog/2018/11/introduction-text-summarization-textrank-python/

https://analyticsindiamag.com/here-are-top-five-text-summarization-tools-that-could-be-helpful/

https://pkghosh.wordpress.com/2019/06/27/six-unsupervised-extractive-text-summarization-techniques-side-by-side/

https://www.kdnuggets.com/2019/01/approaches-text-summarization-overview.html

https://www.upgrad.com/blog/text-summarization-in-nlp/

https://huggingface.co/transformers/pretrained_models.html

https://huggingface.co/transformers/task_summary.html

https://huggingface.co/transformers/main_classes/pipelines.html

https://www.machinecurve.com/index.php/2020/12/21/easy-text-summarization-with-huggingface-transformers-and-machine-learning/

https://www.thepythoncode.com/article/text-summarization-using-huggingface-transformers-python

https://towardsdatascience.com/simple-abstractive-text-summarization-with-pretrained-t5-text-to-text-transfer-transformer-10f6d602c426

https://towardsdatascience.com/fine-tuning-a-t5-transformer-for-any-summarization-task-82334c64c81

https://github.com/dmmiller612/bert-extractive-summarizer

https://medium.com/nerd-for-tech/nlp-with-hugging-face-transformers-a41caadf6f2

https://paperswithcode.com/task/text-summarization

https://paperswithcode.com/paper/attention-is-all-you-need

https://github.com/tensorflow/models/tree/master/official/nlp/transformer

https://towardsdatascience.com/summarize-reddit-comments-using-t5-bart-gpt-2-xlnet-models-a3e78a5ab944

https://towardsdatascience.com/gpt-3-transformers-and-the-wild-world-of-nlp-9993d8bb1314

https://www.toptal.com/deep-learning/exploring-pre-trained-models

https://analyticsindiamag.com/gpt-3-vs-bert-for-nlp-tasks/

https://www.topbots.com/leading-nlp-language-models-2020/

https://huggingface.co/transformers/pretrained_models.html

https://huggingface.co/transformers/model_doc/t5.html

https://www.machinelearningplus.com/nlp/text-summarization-approaches-nlp-example/

https://ai.googleblog.com/2020/03/more-efficient-nlp-model-pre-training.html

https://daleonai.com/transformers-explained

https://ojs.aaai.org/index.php/AAAI/article/view/4724/4602

https://www.sciencedirect.com/science/article/pii/S1877050920306578/pdf?md5=a99d7ef735765aaf134239941acdfa35&pid=1-s2.0-S1877050920306578-main.pdf

https://www.sciencedirect.com/science/article/pii/S1877050920306578

https://github.com/yaushian/Unparalleled-Text-Summarization-using-GAN

https://paperswithcode.com/task/abstractive-text-summarization?page=2

https://arxiv.org/abs/1711.09357

https://www.researchgate.net/publication/340704425_Deep_Text_Summarization_using_Generative_Adversarial_Networks_in_Indian_Languages

https://ieeexplore.ieee.org/document/8576043

https://likicode.com/upload/AAAI_accepted_paper.pdf

https://www.semanticscholar.org/paper/Generative-Adversarial-Network-for-Abstractive-Text-Liu-Lu/2056a1eedc7c0e912b68906e218081df824723c4

https://medium.com/analytics-vidhya/deep-reinforcement-learning-deeprl-for-abstractive-text-summarization-made-easy-tutorial-9-c6914999c76c

https://medium.com/@namita.atri/abstractive-extractive-text-summarization-using-reinforcement-learning-ca1103691533

https://dmkd.cs.vt.edu/papers/SDM19.pdf

https://www.analyticsvidhya.com/blog/2019/06/comprehensive-guide-text-summarization-using-deep-learning-python/

https://openai.com/blog/learning-to-summarize-with-human-feedback/

https://towardsdatascience.com/text-summarization-using-deep-neural-networks-e7ee7521d804

https://aclanthology.org/I17-2033/

https://www.researchgate.net/publication/322715462_Deep_Reinforcement_Learning_for_Extractive_Document_Summarization

https://daneshyari.com/article/preview/6864484.pdf

https://arxiv.org/abs/1810.06667

https://github.com/theamrzaki/text_summurization_abstractive_methods

https://towardsdatascience.com/understanding-electra-and-training-an-electra-language-model-3d33e3a9660d

https://ai.googleblog.com/2020/03/more-efficient-nlp-model-pre-training.html

https://github.com/google-research/electra

https://bijular.medium.com/electra-vs-bert-a-comparative-study-36f07ba88e61

https://openreview.net/forum?id=r1xMH1BtvB

https://towardsai.net/p/nlp/electra-pre-training-text-encoders-as-discriminators-rather-than-generators

https://www.findbestopensource.com/product/google-research-electra

https://arxiv.org/pdf/2006.08218

https://arxiv.org/pdf/2003.10555

https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html

https://huggingface.co/transformers/model_doc/electra.html

https://huggingface.co/transformers/model_summary.html

https://towardsdatascience.com/understanding-electra-and-training-an-electra-language-model-3d33e3a9660d

https://arxiv.org/pdf/2006.05354

https://www.machinelearningplus.com/nlp/text-summarization-approaches-nlp-example/

https://chriskhanhtran.github.io/posts/extractive-summarization-with-bert/

https://www.machinecurve.com/index.php/2020/12/21/easy-text-summarization-with-huggingface-transformers-and-machine-learning/

https://www.topbots.com/key-updates-to-transformer-architecture-2020/

https://www.analyticsvidhya.com/blog/2019/06/comprehensive-guide-text-summarization-using-deep-learning-python/

https://towardsdatascience.com/how-to-perform-abstractive-summarization-with-pegasus-3dd74e48bafb

https://towardsdatascience.com/pegasus-google-state-of-the-art-abstractive-summarization-model-627b1bbbc5ce

https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html

https://signal.onepointltd.com/post/102ghb9/exploring-pegasus-a-new-text-summarization-nlp-model

https://medium.com/thecyphy/generating-abstractive-summaries-using-googles-pegasus-model-18eef8ae985b

https://huggingface.co/transformers/model_doc/pegasus.html

https://jingqingz.github.io/publication/2019-PEGASUS

https://github.com/google-research/pegasus/issues/13

https://arxiv.org/abs/1912.08777

https://github.com/google-research/pegasus

https://colab.research.google.com/drive/1-zq8AJktuC3gQAHTuSiiZ_qvDl4wK7rq#scrollTo=5Zvf8y7fSjOP

https://github.com/openai/gym

https://github.com/deepmind/sonnet

https://medium.com/thecyphy/generating-abstractive-summaries-using-googles-pegasus-model-18eef8ae985b