Anytime you fail to clear a level completely, you will have points deducted from your score for each block left.