Well first of all, for that technology to work (the one that interacts outside of the system), you'll need a second object. Your saying that the bottom half of the system will be the second object. The only problem with that, is that you physically have to manipulate the object in the real world to manipulate it in the digital world. Next you wanted to have the stylus be tracked by the camera so that you could interact with the object. This is feasible but I see a cord being attached to the stylus. The popping out aspect, however, could never work in the holographic type of way you are talking about. As soon as you block your perspective with the stylus, either one or both eyes are not getting information to create the illusion of depth for your brain, and thus the illusion would cease to exist.
It's a great idea, but the combination of technologies that you have in your head (1+1 = 3), just isn't sophisticated enough right now.