In the simplest terms, when you apply one of the ICS leaks from .84 on up, there are some changes made to the internal structure that can not be undone by the .173/.181 fastboot files. Thus certain files fail when applied and you are stuck with parts of .173/.181 and parts of .84/.85/.203/.204... depending on which file you upgraded with and which you are trying to downgrade to. Leaving you with an unusable device.
that makes complete sense, thanks, i appreciate it.