Strange matter wins physics Nobel
They were named at a press conference in Sweden, and join a prestigious list of 200 other Physics laureates recognised since 1901.
The Nobel Committee said the work had "opened the door on an unknown world".
When matter is very cold or extremely flat, scientists start to see unusual behaviour from the atoms.
These phenomena complement the more familiar phases of matter, namely when things change from solid to liquid to gas.
The laureates` discoveries had helped scientists in designing new materials.
Old work, new uses
Prof Haldane commented: "I was very surprised and very gratified."
"The work was a long time ago but it`s only now that a lot of tremendous new discoveries are based on this original work, and have extended it."
All three researchers used maths to explain strange physical effects in rare states of matter, such as superconductors, superfluids and thin magnetic films.
Kosterlitz and Thouless focused on phenomena that arise in flat forms of matter - on surfaces or inside extremely thin layers that can be considered two-dimensional.
This contrasts with the three dimensions (length, width and height) with which we usually describe reality.
Haldane also studied matter that forms threads so thin they can be considered one-dimensional.
Acting chairman of the Nobel committee, Prof Nils Mårtensson, commented: "Today`s advanced technology - take for instance our computers - relies on our ability to understand and control the properties of the materials involved.
"And this year`s Nobel laureates in their theoretical work discovered a set of totally unexpected regularities in the behaviour of matter, which can be described in terms of an established mathematical concept - namely, that of topology.
"This has paved the way for designing new materials with novel properties and there is great hope that this will be important for many future technologies."
Although British in origin, the three individuals all now live and work in the US.
David Thouless was born in 1934 in Bearsden. He is an emeritus professor at the University of Washington.
Duncan Haldane was born in 1951 in London. He is a professor of physics at Princeton University.
Michael Kosterlitz was born in 1942 in Aberdeen. He is currently affiliated to Brown University.