File photo of George Zimmer, founder and executive chairman of The Men's Wearhouse Inc., on May 1, 2012 (© Patrick Fallon-Bloomberg via Getty Images)
Shares of Men's Wearhouse (MW) traded down Wednesday after the specialty retailer abruptly fired founder George Zimmer, who had been the company's TV pitchman since 1985, and postponed its annual meeting, originally scheduled for today.

Though the men's clothier was mum on the specifics behind the ouster of Zimmer, who had been the company's executive chairman, he told CNBC that he has had concerns for months about the strategy the board has undertaken. Zimmer wasn't more specific.

"Instead of fostering the kind of dialogue in the boardroom that has in part contributed to our success, the board has inappropriately chosen to silence my concerns through termination as an executive officer," CNBC quotes Zimmer as saying.

Earlier, Men's Wearhouse had said "the board expects to discuss with Mr. Zimmer the extent, if any, and terms of his ongoing relationship with the company." Again, no specifics were given.

That raises the question of why Men's Wearhouse would want to maintain a relationship with someone whom it had terminated so suddenly. One analyst CNBC quoted said Zimmer had difficulty letting go of the reins of power.

As the chain's co-founder in 1973, Zimmer remains a large stockholder, owning 1.77 million shares, according to the company's latest proxy. That raises the possibility that he'll try to mount some sort of challenge. Until today, Zimmer had a pretty sweet deal. In 2012, his compensation, which included perks such as the personal use of corporate aircraft, totaled $1.09 million.

Men's Wearhouse had the right to use Zimmer's likeness, which he had trademarked, while he was employed by the company in exchange for a $10,000 annual fee. The payoff doesn't end  there for Zimmer, who's known for the catchphrase "You are going to like the way you look -- I guarantee it."

"If Mr. Zimmer ceases to be an employee of the Company for any reason, then we would be required to pay Mr. Zimmer or his estate $250,000 per year for four years for the continued license," the proxy says.

During the most recent quarter, Men's Wearhouse reported better-than-expected​ results. Even with Wednesday's sell-off, the shares are up about 18% this year.

But as Zimmer has discovered, founding a company doesn't come with the ironclad job security that some people  might think, especially when it's publicly traded.

Groupon (GRPN) ousted CEO Andrew Mason earlier this year after investors decided he lacked the skills needed to run the company he'd founded. Best Buy (BBY) founder Richard Schulze was forced out of the electronics retailer last year for failing to tell his fellow board members about CEO Brian Dunn's affair with an employee. Dunn was fired, but Schulze rejoined the retailer this year as chairman emeritus.

Men's Wearhouse hasn't forgotten Zimmer quite yet. His picture is still prominently shown on the company's website, which makes the story of his ouster even stranger.

Jonathan Berr does not own shares of the listed stocks. Follow him on Twitter @jdberr.

More on moneyNOW