Deleting the item from the shopping cart does not delete the order entry/entries (which is probably as intended, but mentioning just in case it isn't)

(Click to enlarge)

I have partly resolved it, however the core issue of duplicate orders is by design and not something easy to solve.

1) Breadcrumb issues - solved, including some I also found in the Admin Zone, along with an issue with a bad <title> on the shopping cart.

2) Also fixed that order cancellation option showed for already cancelled orders in Admin Zone.

3) I have made it smarter about clearing out old orders.
a) If you explicitly cancel your order from the payment gateway, it actually will now do less order cleaning up, only cleaning out orders that are over a week old. It won't actually delete that order now. So this goes against what you are suggesting, but it's important because there's nothing stopping someone clicking back to get back to the payment gateway then deciding to proceed. And that would result in a payment with no matching order, not good!
b) Additionally when old orders are cleaned up, if there are notes on that order it won't now delete them, but cancel them.
c) However, when refreshing the cart it will still create duplicate orders. This is necessary because the order has to be created immediately, as the payment gateway buttons are directly embedded on the cart.
i) It is perfectly possible for someone to open the cart, pay, the payment to be delayed coming back to the site then refresh - so we can't delete the pending order and recreate because it might have a payment coming soon (we can't allow a payment not matched with an order).
ii) ... and we can't just keep the same order open in the cart either for the same reason (we can't allow double purchase by letting them click pay on that order twice).
iii) we could probably try and track that an exact same order exists and reset to an empty cart, or show an error message. But we don't know that existing order was actually carried through to payment.
d) It will however delete pending orders more than a week old when refreshing the cart.

I think the proper solution to '3c' would be to not have payment gateway buttons direct on the cart, but to pass through an interstitial page - which would create the order and clear the cart. However that's adding further complexity, and the cart is meant for very simple use cases. i.e. selling T-shirts and mugs, not running a sophisticated store. It may also be a violation of the T&Cs of payment gateways who want you to use their code and logos in a specific way.

Honestly it's damned complex and tying my head in knots thinking about it.

Just think of any time you going to the cart as the 'checkout' step, and any changes to the contents of the cart, or going back to the checkout, is creating a new order. Just we can't clear out the old one as we can't track what might have happened on the payment gateway end prior to a payment finishing being processed.

I don't intend to use the shopping cart and was just reporting my discoveries when testing. Solution for 3c might be useful for those that do, but it doesn't affect my plans. If it's confusing to you I have no chance :) I just found it odd that I ended up with several orders just by visiting the page after clicking buy (and not proceeding any further). I don't even have a payment gateway set up to proceed beyond the intent to buy.