It is not called Rocket science for nothing
A lunar mission is inherently more complex than a simple Earth-orbit mission; it involves more parts that have to be precisely fit together. Let's outline a typical Earth-orbit mission--flying up to the International Space Station, say:
Liftoff and ascent: That is, getting your rocket off the ground and up into Earth orbit. The timing and trajectory must be precise, or you won't be able to dock with the station. Also involves the various orbital maneuvers you perform on the way up.
Rendezvous and docking: Come up on the station, get close to it, and dock. Only possible if you got the previous step right; even then, it's tricky, because you're trying to avoid colliding with the station, which would not be a desirable outcome. Eventually, you undock, which leads to:
Deorbit, reentry, and landing: Reverses the process by which you got up to the station. The time of your deorbit burn must be precise, to ensure that you enter the atmosphere and land where you're supposed to.
Now, let's consider an Apollo-type lunar mission and all the steps it involves:
Liftoff and ascent: Of course, you're not rendezvousing with a space station this time, you're going up to a "parking orbit" where you can check out your ship prior to heading for the moon. Still, the timing and trajectory are critical here.
Translunar injection (TLI): Fire your rocket again to break orbit and head towards the moon. Timing, again, is critical, to ensure you're actually headed for the moon.
Lunar orbit insertion (LOI): Once you get to the moon, you're actually going too fast to stay there; if you don't slow down, you'll whip around the moon and ideally be headed back towards Earth (a "free-return trajectory"). So you have to burn to slow down and enter a lunar orbit.
LM undocking, deorbit, and landing: Each of these could be a separate bullet point in and of itself. Difficulty here is compounded by the fact that the flight controllers who can look over your shoulder and make sure you're doing it right are far enough away that communications lag is becoming a problem. But, even after you've gotten down on the ground, the process is only half-done...
LM ascent, rendezvous, and docking: Now you've got to fly your lunar module back up to where your CSM is waiting for you in orbit. Again, your launch and ascent must be precisely timed and aimed, or you're not going home.
Trans-Earth injection (TEI): Having slowed down enough to get into lunar orbit, you now have to speed up again to get out of lunar orbit and start heading for home. And it'd better be timed well enough that you are headed for Earth.
Reentry and landing: Substantially different from reentry from an Earth orbit; you're coming in straight from deep space. And the angle at which you come in is very critical: come in too steep and you burn up in the atmosphere and die, but come in too shallow and you skip off the atmosphere like a stone skipping off the surface of a pond, and you don't get home. Plus there's the matter of where you land; Apollo was designed to splash down in an ocean, but you really ought to have a ship somewhere nearby to recover your spacecraft.
Just putting together all the maneuvers for something like this is vastly more complex, never mind designing the spacecraft that have to be able to accomplish all this. And, of course, more complexity means more things that can possibly go wrong
I AM trust in GOD, I AM belief in THYSELF