Giving your horse the old stink-eye is a sure way to get it on your bad side, and research backs that up.

A new study shows they don't only notice when you're in a bad mood, they remember it later, even if you've done nothing to upset them.

Evidence based on research conducted by the Universities of Sussex and Portsmouth suggests horses will respond differently to a person based on whether a previously seen photograph shows them with a smile or a frown.

"What we've found is that horses can not only read human facial expressions but they can also remember a person's previous emotional state when they meet them later that day – and, crucially, that they adapt their behaviour accordingly," says psychologist Karen McComb.

"Essentially horses have a memory for emotion."

For most of us living in a motorised, post-horse world, this is little more than an interesting bit of trivia.

But for people who work alongside these majestic animals it's good to have some solid evidence backing up what many already suspect – horses are paying close attention. So mind your manners.

Really, it should come as little surprise. Other domesticated species of social animal, such as dogs and even the humble sheep, can respond to and remember details of human faces.

To see if horses had a knack for responding to the emotions we wear on our face, the researchers tested them with photographs displaying angry and happy expressions.

They found that the subjects' showed negative reactions and increased heart rate to the more aggressive looking images.

Surprisingly, they also showed a bias in how they looked at the images, turning their head to gaze at it with their left eye, possibly thanks to differences in how their brain's hemispheres process threatening stimuli.

To follow up on this study, the team recently went back to the stable and used the same basic method to test if the response had a lasting impact on how 24 different horses in four locations across Sussex treated the people behind the smiles and snarls.

A few hours after seeing a photograph of an individual grinning or frowning, the horses got to meet the subject in the picture. Only this time the person wore a neutral expression.

It didn't matter – the horses appeared to remember the person and their mood, using different sides of their head to look while displaying an elevated heart rate for those they recalled as looking rather grumpy.

Horses are incredibly intelligent animals that pick up on the smallest shifts in posture.

Early last century a horse named Clever Hans attracted global attention for appearing to recognise numbers and even do arithmetic, while really he was reacting to near imperceptible shifts in his owner's posture during the 'counting' process.

So just to be sure these clever horses weren't being subtly influenced by similar factors, the subjects in the photographs were ignorant as to which pic the horse had seen, preventing them from giving away the game.

A similar experiment was also carried out on another 24 horses serving as controls, which were met with a po-faced stranger rather than the person in the picture.

Following a statistical analysis of the heart rates and behavioural responses, it looked clear that they had a talent for recognising people's emotional expressions hours later.

"We know that horses are socially intelligent animals, but this is the first time any mammal has been shown to have this particular ability," says University of Portsmouth psychologist Leanne Proops.

"What's very striking is that this happened after just briefly viewing a photograph of the person with a particular emotional expression – they did not have a strongly positive or negative experience with the person."

Perhaps the real take home lesson is next time you want to play poker, saddle up and bring along a friend.

This research was published in Current Biology.