In antebellum North Carolina, General Quintard lives a reclusive life on his plantation. When he dies, he has no known heirs. A young boy lived with him, Hannibal Wayne Hazard, and it’s widely suspected that he was some relation, but none knows who the boy is certainly. When Bob Yancy offers to take care of Hannibal, no one offers any objection.
Yancy lives happily with Hannibal, raising him as his own son, until a man appears with a court order for custody of the boy. The town views it with suspicion, but whatever the underlying reason, Yancy has no intention of giving up his “nevvy”. The two flee the state for Tennessee, where they hope to find Betty Malroy, a wealthy woman who took an interest in Hannibal during her brief visit to town.
Along the way, Yancy is accosted at a tavern by Captain Murrell. The boy, believing Yancy to be killed, escapes on his own and finds his way to Judge Slocum Price. The Judge insists that he’s seen better days, and there must be some truth in that, since his present state couldn’t get much worse. He and his dear friend Solomon Mahaffy currently live for little else besides corn whiskey. Still, the Judge has a highly cultivated sense of honor and duty, and so the two set out with Hannibal to find Betty. Along the way, the Judge comes to care a great deal for Hannibal, particularly when he learns that his last name is Hazard.
In Tennessee, Betty, who had be away at school up north, assumes ownership of her plantation, Belle Plain. This greatly disturbs Tom Ware, her half-brother, who, were it not for Betty, stands to inherit the valuable estate. Tom is involved with Murrell, who heads a criminal organization called the Clan that kidnaps slaves from the upper south to resell in the deep south. Murrell, however, has been plotting a new scheme that involves instigating a slave rebellion which will inevitably end in failure, but will greatly increase the value of Murrell’s slaves. Tom has no faith in this scheme, but is willing to work with Murrell because Murrell also wants Betty; if Betty disappears, then Belle Plain is Tom’s.
The Judge reaches Tennessee, but by that point has grown too attached to Hannibal to let him go. He decides to settle in town, aided by Betty, and resume his law practice. When researching a land title at the town office, he stumbles across the name Colonel Fentress, which evidently means a great deal to him. He buys a pair of dueling pistols and begins target shooting.
Betty is kidnapped, and along with her Hannibal, who had been visiting at the plantation. They’re taken across the river and hidden in the overseer’s cabin. The Judge vows to recover both. It’s at this point that Yancy also reappears. He was badly injured and out of his wits for several days, but not killed. The Judge calls on Colonel Fentress, who he knows is in reality David Gatewood, and who years ago robbed him of $30,000 and his wife and daughter. The daughter married into the Hazard family and had a son. This son, through his grandmother, was General Quintard’s only heir. Fentress, the Judge asserts, was behind his abduction because he wanted to claim the Quintard estate, but the Judge now acknowledges that Hannibal is his grandson and he wants him back. Fentress denies everything. The Judge challenges him to a duel, which Fentress accepts, but he conspires to have the Judge detained during the appointed time. Mahaffy takes his friend’s place and is shot dead.
Murrell scheme begins to unravel as his trusted associate, Hues, turns out to be an undercover agent and arrests him. Betty and Hannibal are rescued, but the Clan is a powerful organization and it’s not at all certain if Murrell can be held. Many of the authorities in town flee, but the Judge fearlessly takes possession of the case and charges Yancy with holding Murrell — and to kill him if he tries to escape. Fentress, who by now is a known conspirator with Murrell, appears with an order for his release. The Judge (who, drunk or not, is an actual federal judge) counters by charging Murrell with felony counterfeiting and rearresting him. Fentress reaches for his gun, but the Judge pulls out his own and shoots him in the shoulder. Fentress and the rest of Murrell’s supporters flee. Fentress is found dead some time later, evidently robbed by Tom, who was escaping justice with him. Murrell is found guilty and dies in prison. Hannibal inherits the Quintard estate. The Judge claims Fentress’s property as his own, hiring Yancy as his overseer. The two raise Hannibal together. Mahaffy is burred beneath the Judge’s window. He tells Yancy that he wants to be buried beside him when his time comes.