Dark-circled "panda" eyes and puffy lids can even put others off socialising with you, they say.
People were rated by strangers as less healthy and approachable when they had tired faces.
The experiment
The researchers asked 25 university students, some male and some female, to be the guinea pigs in their sleep experiment.
The volunteers - who were given payment for their help - were sent home with a kit that would measure their night-time movements to check that they had not cheated and slept when they should not have.
They were asked to get a good night's sleep for two consecutive nights.
A week later, they were asked to restrict themselves to only four hours sleep per night for two nights in a row.
The researchers took make-up free photos of the volunteers after both the good and the bad sleep sessions.
Next, they asked 122 strangers - women and men living in Sweden's capital city, Stockholm - to have a look at the photos and rate them on attractiveness, health, sleepiness and trustworthiness, as well as asking them: "How much would you like to socialise with this person in the picture?"
The strangers were good at judging if the person they were looking at was tired, and, if they were sleepy, their attractiveness score suffered.
The strangers also said they would be less willing to socialise with the tired students, who they also perceived to be less healthy, Royal Society Open Science journal reports.
The Karolinska Institute researchers says this makes sense in evolutionary terms.
"An unhealthy-looking face, whether due to sleep deprivation or otherwise, might activate disease-avoiding mechanisms in others."
In other words, people don't want to hang around with people who might be ill, whereas someone who looks energetic and fit will hold lots of appeal.
Lead researcher Dr Tina Sundelin added: "I don't want to worry people or make them lose sleep over these findings though.
"Most people can cope just fine if they miss out on a bit of sleep now and again."
Dr Gayle Brewer, a psychology expert at the University of Liverpool and member of the British Psychological Society, said: "Judgement of attractiveness is often unconscious, but we all do it, and we are able to pick up on even small cues like whether someone looks tired or unhealthy.
"We want our partners to be attractive and energetic.
"This study is a good reminder of how important sleep is to us."
More about: #sleeping
