I think the Native Appropriations blog explains it best. Native Americans have been so terribly abused by the US, and continue to be abused now -- they suffer the worst poverty with the least aid, the US occupies/mines land that by US law belongs to Native peoples, Native women are the most raped group of all US women (with little legal recourse), Native people suffer the most from unemployment when the economy suffers, etc. As a non-Native person, one benefits from that abuse whether they want to or not -- the US government robs from Native peoples to give to "real americans" and if Native peoples lose their jobs first that financially benefits everyone who is non-Native, etc. So wearing "native" dress (in quotes because usually what non-Native people wear is not actually Native dress) highlights one's ignorance about the experiences of Native people and gives the impression (usually correct) that one is ignorant about or ignoring the suffering of Native people. And it is disrespectful to take from someone's culture without understanding the significance of that item, at the very least.
This is not to say that I don't think there is any legitimate way for non-Natives to use/appreciate Native art, or that I think cultures should never mix; I don't believe culture is static or can be truly 'owned.' But I think there is no respectful way to do that WITHOUT self-educating on the suffering that Native peoples face and trying to change that. I do not know what the respectful way would be, but I know the usual way is extremely disrespectful. One cannot say that they are showing respect or honor if they are ignoring the very real, very present suffering of the people they are supposedly "honoring."
where does it end? I think it begins where there is a history of oppression and genocide, especially when that oppression is still very much in practice. Where that is not the case, I don't think there is a clear-cut right and wrong, but the real question is what are you looking for? Are you wanting to see what you can get away with without being scolded by your friends? or are you wanting to know how your actions/dress/etc affect the people from whom you are culturally borrowing? 'cause that's the real question. It is a gross expression of privilege to decide whether or not an oppressed group's objections are legitimate or "silly." It was "pointless/silly" for people of color to want to learn to read and it was "oversensitive" for women to object to sexual harassment, until they had enough of a voice that the dominant class couldn't help but hear. If you can't relate to a group's "silly, oversensitive" objections, educate yourself, listen, stop thinking about what you "should be allowed" to do and start thinking about what your actions mean to groups that do not have your privilege.