The top layer (of dirt) comes up pretty easy, but the next layer (of rock) is more difficult. It was suggested that this was road base from the original construction of the curb.
But the technique is the same -- separate the rocks for disposal, mix the dirt with the rock dust and lots of leaves and keep going. Eventually this gets us all the way to the curb at least 24 inches down.
Doing so exposes the question of what to do next to the sidewalk itself. The "dirt" under the sidewalk is a mixture of rock and dirt and roots and the roots tend to raise the sidewalk.
We would like to discourage roots from going under the sidewalk. So we decided to pour a cement wall along the sidewalk and continue the 6x6 limestone blocks from the end of the retaining wall to the street.
Pouring the cement wall requires framing it up, as before.
We used some left-over rebar in this section of concrete, since we had been unable to get rid of it any other way.
Then, once the concrete is poured, we mortar some of the remaining 6x6 limestone blocks from the retaining wall onto the top of the cement wall.
For this section of the 6x6 limestone blocks, we mixed up our own mortar, from sand and white masonry cement, using a 3 parts of sand to 1 part of cement ratio. This seemed to work okay, but despite wearing gloves, it created severe chemical burns on the backs of my hands. After two weeks those were pretty well healed.
Once the sidewalk wall was in place, the remaining job was simply to move all the dirt that we dug out back into place, mixing the different types of dirt and lots of leaves as we did.
This fills the area, but leaves a wall edge of 4 to 6 inches all around the yard, so we need to bring in about 18 cubic yards of new dirt to fill it up. More as things settle.