I'm not worried. This is the same thing as when 3DS went on sale. Ok, they did do some better numbers (I wonder if it was for the 3D, or just brand loyalty?) Still, EVERY launch console/handheld needs patience to see numbers. The only one that did insane numbers, was the Wii, and because it was this new thing. Also, maybe a Zelda game on launch helped for some.
Seriously, just wait... even if things look bad now, it WILL get better with more titles coming out! Think about it: wouldn't a Call of Duty handheld be a system seller? PSP had one, but that was PSP (and before CoD was the monster it is now).