I'm glad you understood.

I figured out one example to prove that even pushing junk into sun's direction wouldn't help.

1. Gravity and rocket propulsion would be forces acting into the same direction, right?
2. If they are of same direction, we can analyze the gravity of some bigger body instead of the sum of the above mentioned forces. Figuratively speaking, let's just put some bigger star there to catch our junk, which gravitational force would be equal to Sun's plus the force from propulsion.
3. Gravitational pull is proportional to star's mass and reciprocal to distance between objects squared - so we go back to the Sun in the middle of the system, only put our object closer. All these operations we made are equivalent - you still get the same attracting force, acting upon the object.
4. As everyone (except creationists) possibly know, the closer you are to a massive body, the more speed you need to keep away from it. We have put our junk closer to the sun (or replaced the Sun with something bigger), thus it means the speed it had is not sufficient to maintain mostly circular orbit that Earth has.
5. However, it doesn't mean it will not orbit Sun. It will - in an ellyptical orbit. Even worse - no matter how long you will push it into the sun, you will never make it to fall into it. That is because it will start building the speed up from falling, and you will have to increase the push more and more to keep it falling
6. And yes, by replacing propulsion with bigger source of gravity, I made an approximation... in the wrong direction. Gravitational pull increases with distance decreasing - propulsion wouldn't. What follows is a simple conclusion - the orbit wouldn't be even that much ellyptical as with bigger sun.

With all that in mind we can deduce that pushing something into Sun directly is... well, not bright. Especially considering that all that junk would come back one day to Earth after making a full orbit. Not a pretty sight, huh?

The only way I see landing junk into Sun is pushing all that pile of junk not into Sun, but 90 degrees away, into the direction, opposite to which Earth is travelling. Then if we apply constant push into that direction, sooner or later we will slow that junk down considerably and then it will fall into Sun. Well, it's ellyptical orbit will get sharper and sharper until it will pass its atmoshpere, to be more exact.

I hope all this babbling was understandable