Confirmation of delete triggered by Delete key shortcut

My webapp has a list of notes. Deleting a note is comparable to deleting a file. The underlying storage does not currently allow a trash-can paradigm (undoable delete). So, deleting a note is a two-step process. Using a mouse or touchscreen, the user long-presses to slide in a toast-like overlay, then clicks the Delete button in the overlay:

I'd like to allow a keyboard shortcut for delete, so typing the Delete or Backspace key slides in the overlay. The question is: How should the user confirm this using the keyboard?

Allowing a second Delete or Backspace key to confirm is risky - if the user thinks they're in the note editor, they might type Delete or Backspace repeatedly to delete some text they're looking at.

I've tried setting the focus to the Delete button when the user types Delete or Backspace, so the user confirms with Enter or Space. Unfortunately, that interferes with other keyboard shortcuts, such as up and down arrows.

Currently, I allow typing Enter after Delete or Backspace, to confirm the deletion, but that's far from obvious. Is there a better solution?