In 2005, in the aftermath of Hurricane Rita, Fox News correspondent Geraldo Rivera stood in downtown Port Arthur, Texas, dwarfed by empty streets and row upon row of abandoned buildings.

"He was here saying, 'Look at what Rita did to this town. It's gone, just gone!'" said Melvin Branch, a local who was watching on TV and wincing.

The hurricane had indeed cut a brutal path through this Gulf Coast community. But it could not be blamed for stripping the former boomtown to a ghostly shell of what it once was. Years of neglect had already done that.

"It was all gone decades before the storm," said Branch, who had returned to his hometown after a long career away. "It is really depressing."

(Article continues after video.)

In some ways, Port Arthur is like hundreds of American towns, cleared out by the decline of industry and the gutting of mom-and-pop stores. But Port Arthur tells another story: one where industry has thrived to the beat of the marketplace, while a community allowed itself to be left behind.

Prosperity and poverty

In the heart of the oil coast, Port Arthur is home to 13 refineries and chemical plants, responsible for producing a fifth of the nation's refined oil -- mostly gasoline, diesel and jet fuel.

In the past five years, three of the city's refineries have undertaken major expansions to add the ability to process thick, sour crude from Canadian oil sands. Motiva Enterprises, which spent $10 billion on its expansion, is now the largest refinery in North America, able to churn out 600,000 barrels a day.

For the community, this means thousands of high-paying jobs, making it the envy of other towns. Average pay for skilled labor in a refinery is about $34 an hour.

"We meet council people from all over the state of Texas," said Harold Doucet, a Port Arthur councilman. "When we say Port Arthur, the first thing they tell us is, 'Oh man, y'all ain't got no problem with money down there.' 'Cause we've got all this industry. We've got jobs. We're rolling in dough."

But outside the refineries, options for good jobs are few. Port Arthur's unemployment rate is routinely parked in the double digits, consistently higher than that of the greater region, state or nation. More than one in five families lives below the poverty line ($23,050 for a family of four), relying on food stamps to get by. The population, now at 53,000, has been dropping fast. The air quality, while improved in recent years, is the source of long-term health problems; cancer rates here are 20% higher than the state average.

Downtown Port Arthurr © Microsoft

Downtown Port Arthur

Voter turnout is dismally low, often less than 20% for City Council races. In a 50-square-block area downtown, the only buildings that appear to be in use are City Hall, the police station, a Gulf Coast museum and, since 2010, a community training center. There's no place to buy groceries or meet a neighbor for a cup of coffee or a beer.

When Hilton Kelley returned to his hometown of Port Arthur in 2000, after a 16-year absence, he felt like he was in the twilight zone.

"To know that we have the largest oil refinery in the Northern Hemisphere yet one of the most dilapidated communities on its borders is just unimaginable," said Kelley, who founded the Community In-Power & Development Association to improve air quality and economic conditions. "I've been in communities where they don't have the industries that make half the money that these companies do, yet their communities are thriving."