Two Tesla Model S cars are displayed at a Tesla showroom in Palo Alto, Calif. © Justin Sullivan, Getty Images

Last year's blistering rally in shares of smaller companies may have cooled off, but many investors say the group still appears pricey.

In recent weeks, investors have pulled back from the riskier corners of the stock market, and small-company stocks are typically more vulnerable to wide price swings than are large company shares. Investors say they are skeptical that small-company stocks will be able to live up to earnings expectations, especially if the U.S. economy continues to struggle to gain momentum.

The result has been two sharp selloffs of small-company stocks this year, yet investors worry that valuations remain lofty and the stocks remain vulnerable to more declines.

"It's hard to find a bargain out there" among small-company stocks, said John Roth, who manages $3.3 billion in the Fidelity New Millennium Fund.

During the first quarter, Roth trimmed small-stock holdings and added to larger stocks he sees as cheaper in the tech space. That left small caps as 12 percent of the fund's portfolio at the end of February, according to the most recent data available, below the 19 percent stake the fund averaged from 2008 through 2013.

In the first two months of this year, he added to his position in International Business Machines (IBM) and trimmed from his holdings of Tesla Motors (TSLA), according to Morningstar.

After the recent declines, Roth said he is weighing whether to buy more of a select few of his favorite small-stock names, but he is being careful about it, because he said prices remain relatively high.

There isn't a precise definition of a small-cap stock, but most investors consider companies with a market capitalization of $1 billion or less to be a small-cap name. The biggest stock in the iShares Russell 2000 Growth (IWO) exchange-traded fund, American Realty Capital Properties (ARCP), has a market valuation of $10 billion, while the smallest have market values below $20 million, according to FactSet.

Last year, the Russell 2000 index of small-company stocks soared 37 percent as the S&P 500 ($INX) rallied 30 percent. Among the highfliers, jeweler Zale (ZLC) is up 378 percent in the past 12 months, clean-energy company GT Advanced Technologies (GTAT) has risen 340 percent, and Rite Aid (RAD) has climbed 175 percent.

However, this year, the Russell 2000 is down 3.5 percent while the S&P 500 has gained 0.8 percent. Even the Nasdaq Composite Index ($COMPX), which took a drubbing from a dive in biotechnology and technology stocks, performed better than the Russell, declining 2.4 percent.

Last week, when the S&P 500 fell less than 0.1 percent, the Russell lost 1.3 percent. Software company CommVault Systems (CVLT) dove 27 percent, advertising-technology company Rocket Fuel (FUEL) fell 13 percent, and business-services provider WageWorks (WAGE) shed 11 percent.

The biggest issue, investors say, is that valuations of small stocks remain rich. The Russell 2000 is at trading around 19 times the expected earnings of its components for the next year, according to Russell Investments. At the start of 2013, the earnings multiple on the Russell was 15.1.

Current valuations are well above their long-term averages. Since the start of 1994, small caps have traded at an average price/earnings multiple of 16.9, according to Russell. Over the last five years, the average has been 15.8.

There is caution even among some fund managers whose portfolios are dedicated to shares of small companies.

Jim Larkins, who manages $240 million in the Wasatch Small Cap Value Fund (WMCVX), is keeping the cash level in his fund toward the upper end of its historical range. As of the end of March, the fund held 7 percent in cash. Over time, the fund has tended to hold between 2 and 8 percent in cash, he said.

"Small caps have had just an amazing run," he said. "Against that backdrop, to see small-caps trailing a little bit, maybe it's about time."

Larkins has been buying shares of some stocks that have been hit in the broader selloff of high-growth technology companies, but he is steering clear of what he sees as high-profile, pricey names, he said. At the end of last year, he trimmed some of his holdings in CARBO Ceramics (CRR), an oil-field-services company, according to Morningstar. It was trading at nearly 26 times its earnings forecasts for the next 12 months after a 49 percent gain in 2013, according to Morningstar.

"The real darlings aren't even close to value territory," he said.

However, individual investors haven't yet soured on small U.S. stocks. Traditional mutual funds investing in small caps have taken in $3.1 billion this year, according to Morningstar, on top of the $16.4 billion last year.

Carina Diamond, an investment adviser managing $400 million with Cetera Financial in Cleveland, said last year's big gains have had some clients asking whether they should own more small caps.

"They're coming because they see that an asset category has done well, and it's natural that they want to pile into it, with a herding mentality," she said.

Instead, she is trimming her exposure to the small-cap mutual funds she picks for her clients and adding to large-cap mutual funds.

Beyond valuations, another worry for some is the potential for higher interest rates should the Federal Reserve continue to pull back its easing efforts.

"Small caps don't have the credit quality that the larger names command," said Bryan Novak, senior managing director for Astor Investment Management.

The $700 million Astor Long/Short fund, managed by Novak, held 5 percent of its portfolio in small-cap and microcap stocks at the end of February, according to Morningstar. He decided not to add to small stocks through buying more of the iShares Russell 2000 ETF (IWM) earlier this year, he said, because he thought small stocks were likely to be more volatile in 2014 and didn't offer much potential to outperform their larger peers. Instead, the fund bought the Guggenheim S&P 500 Pure Value (RPV) fund, which invests in large company stocks deemed cheap based on several metrics.

Broadly, Novak said that the outperformance of larger stocks and defensive stock plays, such as utilities, made him uneasy about the market's short-term outlook. That, he said, only added to his lack of enthusiasm for small caps.

"When you see a rally that's led by large-cap names, that tends to be a little disconcerting," he said. "To get really positive on (stocks), we'd like to see some more outperformance in small names."

Click here to become a fan of MSN Money on Facebook

More from The Wall Street Journal