We have provided a walkthrough example of Text Summarization with Gensim. Today, we will provide an example of Text Summarization using transformers with HuggingFace library. The theory of the transformers is out of the scope of this post since our goal is to provide you a practical example.
For this example, we will try to summarize the plot from the Fight Club movie that we got it from Wikipedia Movie Plot dataset and we also worked on it for the GloVe model.
Let’s have a look at the long plot:
The unnamed Narrator is a traveling automobile recall specialist who suffers from insomnia. When he is unsuccessful at receiving medical assistance for it, the admonishing doctor suggests he realize his relatively small amount of suffering by visiting a support group for testicular cancer victims. The group assumes that he, too, is affected like they are, and he spontaneously weeps into the nurturing arms of another man, finding a freedom from the catharsis that relieves his insomnia. He decides to participate in support groups of various kinds, always allowing the groups to assume that he suffers what they do. However, he begins to notice another impostor, Marla Singer, whose presence reminds him that he is attending these groups dishonestly, and this disturbs his bliss. The two negotiate to avoid their attending the same groups, but, before going their separate ways, Marla gives him her phone number.
On a flight home from a business trip, the Narrator meets Tyler Durden, a soap salesman with whom he begins to converse after noticing the two share the same kind of briefcase. After the flight, the Narrator returns home to find that his apartment has been destroyed by an explosion. With no one else to contact, he calls Tyler, and they meet at a bar. After a conversation about consumerism, outside the bar, Tyler chastises the Narrator for his timidity about needing a place to stay. Tyler requests that the Narrator hit him, which leads the two to engage in a fistfight. The Narrator moves into Tyler’s home, a large dilapidated house in an industrial area of their city. They have further fights outside the bar on subsequent nights, and these fights attract growing crowds of men. The fighting eventually moves to the bar’s basement where the men form a club (“Fight Club”) which routinely meets only to provide an opportunity for the men to fight recreationally.
Marla overdoses on pills and telephones the Narrator for help; he eventually ignores her, leaving his phone receiver without disconnecting. Tyler notices the phone soon after, talks to her and goes to her apartment to save her. Tyler and Marla become sexually involved. He warns the Narrator never to talk to Marla about him. More fight clubs form across the country and, under Tyler’s leadership (and without the Narrator’s knowledge), they become an anti-materialist and anti-corporate organization, Project Mayhem, with many of the former local Fight Club members moving into the dilapidated house and improving it.
The Narrator complains to Tyler about Tyler excluding him from the newer manifestation of the Fight Club organization Project Mayhem. Soon after, Tyler leaves the house without notice. When a member of Project Mayhem is killed by the police during a botched sabotage operation, the Narrator tries to shut down the project. Seeking Tyler, he follows evidence of Tyler’s national travels. In one city, a Project Mayhem member greets the Narrator as Tyler Durden. The Narrator calls Marla from his hotel room and discovers that Marla also believes him to be Tyler. Tyler suddenly appears in his hotel room, and reveals that they are dissociated personalities in the same body. When the Narrator has believed himself to be asleep, Tyler has been controlling his body and traveling to different locations.
The Narrator blacks out after the conversation, and when he awakes, he uncovers Tyler’s plans to erase debt by destroying buildings that contain credit card companies’ records. The Narrator tries to warn the police, but he finds that these officers are members of the Project. He attempts to disarm the explosives in a building, but Tyler subdues him and moves him to the uppermost floor. Held at gunpoint by Tyler, the Narrator realizes that, in sharing the same body with Tyler, he himself is actually in control holding “Tyler’s” gun. The Narrator fires it into his own mouth, shooting through the cheek without killing himself. Tyler collapses with an exit wound to the back of his head, and the Narrator stops mentally projecting him. Afterward, Project Mayhem members bring a kidnapped Marla to him, believing him to be Tyler, and leave them alone. Holding hands, the Narrator and Marla watch as the explosives detonate, collapsing many buildings around them.
Install the Transformers
For this example we will work with the colab. Let’s install the required packages.
!pip install transformers !pip install torch
Example of Text Summarization with Transformers
Now we are ready to start. Notice that the Transformers provide many pipeline such as:
`~transformers.FeatureExtractionPipeline`.
– `”sentiment-analysis”`: will return a `~transformers.TextClassificationPipeline`.
– `”ner”`: will return a `~transformers.TokenClassificationPipeline`.
– `”question-answering”`: will return a `~transformers.QuestionAnsweringPipeline`.
– `”fill-mask”`: will return a `~transformers.FillMaskPipeline`.
– `”summarization”`: will return a `~transformers.SummarizationPipeline`.
– `”translation_xx_to_yy”`: will return a `~transformers.TranslationPipeline`.
– `”text2text-generation”`: will return a `~transformers.Text2TextGenerationPipeline`.
– `”text-generation”`: will return a `~transformers.TextGenerationPipeline`.
– `”zero-shot-classification:`: will return a `~transformers.ZeroShotClassificationPipeline`.
– `”conversation”`: will return a `~transformers.ConversationalPipeline`.
In our case we will work with the summarization which takes the following parameters.
Summarize news articles and other documents.
This summarizing pipeline can currently be loaded from ~transformers.pipeline using the following task
identifier: "summarization".
The models that this pipeline can use are models that have been fine-tuned on a summarization task, which is
currently, 'bart-large-cnn', 't5-small', 't5-base', 't5-large', 't5-3b', 't5-11b'. See the up-to-date
list of available models on huggingface.co/models <https://huggingface.co/models?filter=summarization>__.
Usage:
# use bart in pytorch
summarizer = pipeline("summarization")
summarizer("An apple a day, keeps the doctor away", min_length=5, max_length=20)
# use t5 in tf
summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base", framework="tf")
summarizer("An apple a day, keeps the doctor away", min_length=5, max_length=20)
from transformers import pipeline
mysummarization = pipeline("summarization")
# Define the input
mytxt = """ The unnamed Narrator is a traveling automobile recall specialist who suffers from insomnia. When he is unsuccessful at receiving medical assistance for it, the admonishing doctor suggests he realize his relatively small amount of suffering by visiting a support group for testicular cancer victims. The group assumes that he, too, is affected like they are, and he spontaneously weeps into the nurturing arms of another man, finding a freedom from the catharsis that relieves his insomnia. He decides to participate in support groups of various kinds, always allowing the groups to assume that he suffers what they do. However, he begins to notice another impostor, Marla Singer, whose presence reminds him that he is attending these groups dishonestly, and this disturbs his bliss. The two negotiate to avoid their attending the same groups, but, before going their separate ways, Marla gives him her phone number.
On a flight home from a business trip, the Narrator meets Tyler Durden, a soap salesman with whom he begins to converse after noticing the two share the same kind of briefcase. After the flight, the Narrator returns home to find that his apartment has been destroyed by an explosion. With no one else to contact, he calls Tyler, and they meet at a bar. After a conversation about consumerism, outside the bar, Tyler chastises the Narrator for his timidity about needing a place to stay. Tyler requests that the Narrator hit him, which leads the two to engage in a fistfight. The Narrator moves into Tyler’s home, a large dilapidated house in an industrial area of their city. They have further fights outside the bar on subsequent nights, and these fights attract growing crowds of men. The fighting eventually moves to the bar’s basement where the men form a club (“Fight Club”) which routinely meets only to provide an opportunity for the men to fight recreationally.
Marla overdoses on pills and telephones the Narrator for help; he eventually ignores her, leaving his phone receiver without disconnecting. Tyler notices the phone soon after, talks to her and goes to her apartment to save her. Tyler and Marla become sexually involved. He warns the Narrator never to talk to Marla about him. More fight clubs form across the country and, under Tyler’s leadership (and without the Narrator’s knowledge), they become an anti-materialist and anti-corporate organization, Project Mayhem, with many of the former local Fight Club members moving into the dilapidated house and improving it.
The Narrator complains to Tyler about Tyler excluding him from the newer manifestation of the Fight Club organization Project Mayhem. Soon after, Tyler leaves the house without notice. When a member of Project Mayhem is killed by the police during a botched sabotage operation, the Narrator tries to shut down the project. Seeking Tyler, he follows evidence of Tyler’s national travels. In one city, a Project Mayhem member greets the Narrator as Tyler Durden. The Narrator calls Marla from his hotel room and discovers that Marla also believes him to be Tyler. Tyler suddenly appears in his hotel room, and reveals that they are dissociated personalities in the same body. When the Narrator has believed himself to be asleep, Tyler has been controlling his body and traveling to different locations.
The Narrator blacks out after the conversation, and when he awakes, he uncovers Tyler’s plans to erase debt by destroying buildings that contain credit card companies’ records. The Narrator tries to warn the police, but he finds that these officers are members of the Project. He attempts to disarm the explosives in a building, but Tyler subdues him and moves him to the uppermost floor. Held at gunpoint by Tyler, the Narrator realizes that, in sharing the same body with Tyler, he himself is actually in control holding “Tyler’s” gun. The Narrator fires it into his own mouth, shooting through the cheek without killing himself. Tyler collapses with an exit wound to the back of his head, and the Narrator stops mentally projecting him. Afterward, Project Mayhem members bring a kidnapped Marla to him, believing him to be Tyler, and leave them alone. Holding hands, the Narrator and Marla watch as the explosives detonate, collapsing many buildings around them."""
# Get the Summary
mysummary = mysummarization(mytxt)
mysummary[0]['summary_text']
Output:
The unnamed Narrator is a traveling automobile recall specialist who suffers from insomnia . He meets Tyler Durden, a soap salesman with whom he begins to converse after noticing the two share the same kind of briefcase . They have further fights outside the bar on subsequent nights, and these fights attract crowds of men . More fight clubs form across the country and, under Tyler’s leadership, they become an anti-materialist and anti-corporate organization .Let’s have a look at the summary:
“The unnamed Narrator is a traveling automobile recall specialist who suffers from insomnia . He meets Tyler Durden, a soap salesman with whom he begins to converse after noticing the two share the same kind of briefcase . They have further fights outside the bar on subsequent nights, and these fights attract crowds of men . More fight clubs form across the country and, under Tyler’s leadership, they become an anti-materialist and anti-corporate organization .”
Not bad!
We could have tried other models like t5-base etc and to compare their outputs. Feel free to play with the library!
