Last night during a D&D session, our GM was discussing how he wanted to throw an encounter at us that could have the potential to be quite easy, or could have the chance to wipe out the party depending on what actions were taken. Ultimately he decided to hold off on that encounter idea until later, but it made me think a little more into how much peril should characters encounter. The DMG explains that 5% of your encounters should be overpowering, but there really isn't a great way to explain that out. Sure it has a definition of level of the encounter versus the level of the party, but it really doesn't give great insight into how big an impact could result from the encounter. Will your party mob a more powerful enemy and will the shear amount of damage make the encounter too easy? On the other hand, will throwing a bunch of enemies at the party cause them to get mobbed and as a result cause characters to die?
Presumably you could answer these questions with some level of ease based on your own DM style. Personally I find myself to be a DM that lets the party take some control of the campaign and allow some fluidity to where they travel, who they meet, and as a result, their actions that they choose will come with consequences, but generally I will try to give the characters a fair shot at surviving a more dangerous encounter. As a result, when I plan for encounters I tend to make it so whatever the party is facing can be dealt with. I could easily throw some monsters that exploit the weaknesses of the party, but that doesn't always instill the right emotion on combat. If I do choose to exploit a weakness, I tend to hold back a little on the challenge of the encounter a bit as well such as to not wipe the party out.
As a DM I have only had one player die, but that was a result of a more risky move which ultimately lead to the character's demise. The struggle that I have with PCs dying is that it is somewhat awkward introducing a new character to a campaign, especially if the party has been together for a while, but PC deaths happen. However, when they do happen though it does open the doors for resurrection adventures/side-quests.
While challenging a party is part of being a DM, it can be difficult to find the right balance to challenge your party without slaughtering them. Ultimately it could require a test encounter to see if something is too powerful or perhaps some creative DM'ing to give the players an out if the challenge is in fact a little to strong. Unfortunately there isn't a formula for this because the possibilities to any encounter are really to endless to calculate.