Yes, if you want to delete the subtitles you just need to go to the subtitle editor and click the small arrow button next to ‘mark complete’ and you’ll be presented with a menu like below. Click ‘Delete Subtitles’. 

Note that this will remove all your work and reset the subtitling status to ‘not yet started’.