DuoRC contains 186,089 unique question-answer pairs created from a collection of 7680 pairs of movie plots where each pair in the collection reflects two versions of the same movie.
DuoRC pushes the NLP community to address challenges on incorporating knowledge and reasoning in neural architectures for reading comprehension. It poses several interesting challenges such as:
| What | 36.92% | Who | 41.22% |
| Why | 2.48% | When | 2.86% |
| How | 6.32% | Reason | 1.23% |
| Boolean | 0.81% | Which | 1.93% |
| Where | 9.18% |
Please cite our work as follows
In the countryside outside the fictional town of Lahood, California, sometime around 1880, [[ thugs working for big-time miner Coy LaHood ride in and destroy the camp of a group of struggling miners] Q1 and their families who have settled in nearby Carbon Canyon and are panning for gold there. In leaving, they also shoot the little dog of fourteen-year-old Megan Wheeler] Q15. As Megan buries her dog in the woods and prays for a miracle, a stranger passes by heading to the town on horseback.
[ Megan’s mother, Sarah] Q16, is being courted by [ Hull Barret, the leader of the miners] Q17 . . . [ Coy LaHood’s son Josh] Q8 . . . [ Club, who with one hammer blow smashes a large rock] Q7 . . . [ Coy LaHood has been away in Sacramento] Q9 . . . [[ Megan, who has grown fond of the Preacher, goes looking for him, but Josh confronts and attempts to rape her ] Q11, while his cohorts look on and encourage him, except for Club, who sees what is happening and moves forward to help Megan] Q13 before Josh can do anything serious. At this moment the [ Preacher arrives on horseback armed with a Remington Model 1858 revolver he has recovered from a Wells Fargo office and, after shooting Josh] Q14 . . . [ Stockburn, who appears startled and says that he sounds like someone that he once knew, but that couldn’t be, since that man is dead] Q5. [ Stockburn and his men gun down Spider Conway] Q4, . . .
[ The Preacher and Hull go to LaHood’s strip mining site and blow it up with dynamite] Q6. [ To stop Hull from following him, the Preacher then scares off Hull’s horse] Q3 and rides into town alone. . . [ Coy LaHood, watching from his office] Q10, . . . [ snow-covered mountains] Q2. [ Megan then drives into town and shouts her love to the Preacher] Q12 and thanks after him. The words echo along the ravine that he is traversing.
Somewhere in California, at the end of the Gold Rush, several horsemen come riding down from the nearby mountains . . . [ The horsemen shoot cattle and Megan’s dog] Q4,Q15, and then chase donkeys as they leave . . .
Hull describes the fight between the stranger and McGill and his men. [ Megan’s mother, Sarah] Q16 says he sounds no different from McGill, Tyson, or any of LaHood’s roughnecks . . . Preacher says there is lot of sinners around, that he can’t leave before he finishes his work. [ Josh says, ”Club”, who gets down and walks into the stream. Everyone is apprehensive. He rolls down his sleeves, and then. . . quickly grabs Hull’s sledgehammer with one hand and strikes the boulder once, screaming, splitting it] Q7 . . .
[ A train pulls into the station from Sacramento while Josh and McGill wait. [ Josh’s father Coy LaHood]] Q8,Q9 (Richard Dysart) exits the train, and then he goes with Josh and McGill. . .
Josh asks what she really came for. She replies that she’s just riding, taking a look around. [ Josh says he wants to take a look too, at her real close. He pulls her off the horse. She screams as he carries her downhill. . . Josh grabs her hair and kisses her. They both fall to the ground. The men cheer him on while Megan begs him to stop] Q11 . . . a gunshot sounds out. Josh gets up and everyone turns around. [ Preacher, on his horse . . . His gun is trained on Josh. Megan sees him and smiles] Q13, . . . [ Josh falls to the ground. He reaches for his gun, but Preacher shoots his hand] Q14 . . . LaHood replies, ”Tall. Lean. His eyes. . . his eyes. Something strange about em. That mean something to you?” [ Stockburn says that it sounds like a man he knew, but that man is dead] Q5 . . . [ LaHood watches through the window] Q10 as they kill Spider, . . .
Hull insists on going with him so Preacher agrees. [ They go to the LaHood camp and blow up their pipes, sluices, tents, and the barracks with dynamite] Q6. [ After fooling Hull to dismount, Preacher scares away his horse. He then tells Hull to take care of Sarah and Megan, and rides into town] Q3 . . . Blankenship tells her that the horses are exhausted and she would kill them. [ Megan runs to the end of town and shouts out thank you to Preacher, that they love him, that she loves him ] Q12 . . . The final shot of the movie shows Preacher riding through the [ snow in the mountains] Q2.
| Question | Shorter Plot Answer | Larger Plot Answer |
|---|---|---|
| For which big-time miner are the thugs who destroyed miners camp in Carbon Canyon working for? | Coy Lahood | Coy Lahood |
| How are the mountains in the film? | Covered with snow | Snow-capped |
| How does the Preacher stop Hull from following him? | Scares Hulls' horse | To stop Hull from following him, the Preacher then scares off Hull's horse and rides into town alone |
| In the movie, who do Stockburn and his men gun down? | Spider Conway | Megan's dog and cattle |
| In the movie, why does Stockburn say that the Preacher could not be the man he once knew? | That man is dead | The man Stockburn once knew is dead |
| What did they use to blow up the strip mining site? | Dynamite | Dynamite |
| What does Club smash? | A rock | A Boulder |
| What is Coy Lahood's relation to Josh? | Father and son | Father |
| Where has Coy Lahood been living? | Sacramento | Sacramento |
| Where was Coy watching from? | Office | A window |
| Who attempts to rape Megan? | Josh | Josh |
| Who does megan love? | The preacher | The preacher |
| Who prevents Josh from raping Megan? | Club | The preacher |
| Who shoots Josh in the hand? | Preacher | The preacher |
| Whose little dog did the thugs shoot? | Megan Wheeler | Megan |
| Who is Megan's mother? | Sarah | Sarah |
| Who is the leader of the miners? | Hull Barret | Coy LaHood |
[ In 1909] Q6, [ there is a raid on the McCandles family . . . Martha, the head of the family . . . [In consequence, she sends for her estranged husband, the aging Jacob ”Big Jake” McCandles] Q9, . . . [ the ransom to the kidnappers, a million dollars] Q4 . . .
[ The Texas Ranger captain is present and offers the services of his men] Q11, . . . Jake, preferring the old ways, has followed on horseback, accompanied by an old Apache associate, Sam Sharpnose. [ He is now joined by his sons, Michael and James] Q2, . . . Knowing that they have been followed by another gang intent on stealing the strongbox, [ Jake sets a trap for them and they are all killed] Q16. [ During the attack, the chest is blasted open] Q1, [ revealing clipped bundles of newspaper instead of money] Q5 . . .
A thunderstorm breaks and [ Pop Dawson, one of the outlaws, arrives to give them the details of the exchange] Q7 . . . [ Jake arranges for Michael to follow after them to take care of the sharp] . . . [ Jake tosses the key of the chest to Fain, who opens it up to discover that he has been tricked] Q12. [ Fain orders his brother Will to kill the boyQ13 but he is shot by Jake. [ Dog is wounded by the sniper] Q15 and Jake is wounded in the leg before Michael kills him. Jake tells the boy to escape but Little Jake is hunted by the machete wielding [ John Goodfellow, who has already hacked Sam to death] Q10 . . . [ With Little Jake rescued, and the broken family bonded, they prepare to head home] Q3.
[[ Jacob McCandles (John Wayne) is a big man with a bigger reputation. A successful rancher and landowner] Q14, his many businesses keep him conveniently away from his home and estranged wife Martha (Maureen O’Hara)] Q9. . . , [ and is demanding one million dollars for his safe return] Q4. . .
[ The local sheriff has convened a posse complete with then state-of-the-art automobiles. [ Two of Jake’s sons, the passionate, gunslinging James (Patrick Wayne) and the motorcycle-riding, sharpshooting Michael (Christopher Mitchum)] Q2 elect to go with the sheriff’s posse. Big Jake decides to set off across the rough terrain on his horse with his Dog at his side, and soon meets up with his Native American friend, Sam Sharpnose (Bruce Cabot), who has brought additional horses and supplies] Q11.
[ They then devise their strategy: James will go have a good time in the saloon, Big Jake will head to the barbershop for a shower, Sam will secrete himself on the roof of the hotel, seemingly leaving Michael alone protecting the strong box. Big Jake tells Sam to listen for a ”disturbance” in the street, and use the distraction to join Michael in the hotel room to protect the strong box. As Big Jake predicted, the gang tries to hit the strong box when it looks most vulnerable. Fain and another of his gang members start a fight with James in the saloon, one keeps a gun on Big Jake in the barbershop, and two others come up the hotel stairs and toward the room. Big Jake dispatches his captor in the barbershop, James fights his way out of the saloon with Jake’s help, and the two head to the hotel. At the hotel, once Sam hears the fight in the saloon, he climbs over the roof and slips in the window to aid Michael in protecting the strong box. Shotguns blast as the gang hits the hotel room. When James and Jake arrive they find Sam, Michael and the Dog unharmed, [ but the strong box has suffered damage. To their horror, James and Michael realize they’ve been risking their lives to protect a box of newspaper clippings!]] Q1,Q5,Q16 . . .
[ Big Jake takes the few moments he has to plan with his sons. He tells Michael of the sharpshooter and instructs him to find a high position and take him out whenever he can. Big Jake takes the Dog and goes to the meet as instructed, while the others follow discreetly behind] Q8 . . . [ As Fain unlocks the strong box, he realizes he´s been had] Q12. Big Jake whispers to him that no matter what happens, Fain will be the first one to die. [Fain screams his command to kill the boy] Q13, . . .
[ Dog giving his life protecting Little Jake from one of Fain’s machete-wielding gang] Q15. . . Sam points him toward James at the exit, [ who helps Little Jake escape. Fain and Big Jake are in a duel to the death, when Michael takes a fatal shot at Fain, saving his father and Little Jake. After a harrowing journey and a risky gamble, the family leaves, happy to be together] Q3.
| Question | Shorter Plot Answer | Larger Plot Answer |
|---|---|---|
| How was the strongbox opened? | It was blasted during an attack | It was damaged during the fight |
| What are the names of Jake's sons? | Jake's sons are Michael and James | James and Michael |
| What did the family prepare to do once Jake had been rescued? | Head home | Leave |
| What is the amount of the ransom? | The ransom amount is a million dollars | One million dollars |
| What was in the strongbox? | Clipped bundles of newspaper | Newspaper clippings |
| What year does the movie take place? | 1909 | No Answer |
| Which outlaw gives details of theexchange to the others? | Pop Dawson | No Answer |
| Who does Jake arrange to follow the restof the group? | Michael | Michael |
| Who is married to Big Jake? | Martha | Martha |
| Who killed Sam? | John Goodfellow | No Answer |
| Who offers his services to help Jakecombat the kidnappers? | The Texas ranger captain offers theservices of men | The posse, native American friend and his two sons |
| Who opens the chest to discover he hasbeen tricked? | Fain | Fain |
| Who orders Will to kill the boy? | Fain | Fain |
| Who owns the ranch? | McCandles family | McCandles family |
| Who wounded the dog? | A sniper | Fain's machete-wielding gang |
| Why does Jake set a trap and kill anothergang? | They were intent on stealing the strongbox | To protect the strongbox |
The submission zip should contain 2 JSON files. One containing the answers for Self RC test set and the other for Paraphrase RC test set. The format of the files should be as follows:
| Date | Team Name | Affiliation | Accuracy | F1 | Model Details | Paper | Code |
|---|---|---|---|---|---|---|---|
| Apr 24, 2018 | DuoRC Authors | IBM Research + IIT Madras | 37.53 | 50.56 | Preprocessing + span identification with BiDAF | Link | Link |
| Apr 24, 2018 | DuoRC Authors | IBM Research + IIT Madras | 15.31 | 24.05 | Preprocessing + Data augmentation + Span identification with BiDAF + Query-based abstractive summarizer | Link | Link |
| Date | Team Name | Affiliation | Accuracy | F1 | Model Details | Paper | Code |
|---|---|---|---|---|---|---|---|
| Apr 24, 2018 | DuoRC Authors | IBM Research + IIT Madras | 14.92 | 21.53 | Preprocessing + span identification with BiDAF | Link | Link |
| Apr 24, 2018 | DuoRC Authors | IBM Research + IIT Madras | 5.42 | 9.64 | Preprocessing + Data augmentation + Span identification with BiDAF + Query-based abstractive summarizer | Link | Link |
You can find the train, development and test sets here