good stuff. i'm glad i riled someone enough to elicit an opposing view. =)
ultimately, it's purely asthetics. i happen to like 4'33", einstein on the beach, williams mix, threnody to the victims of hiroshima and lots of other pieces in those genres. do i listen to them constantly? no. but that doesn't mean i don't like them nor that i haven't grown as a musician and person by listening to them and trying to understand them. so, before even beginning this post, i'm admitting that ultimately it's futile to even respond to your post -- except to say, "ok, you don't like that stuff, but i do."
that being said. what i *can* do is explain *why* i like those pieces. i would classify those songs as conceptual art, and so i offer my defense not of any specific piece, but of conceptual art, in general. people in all endeavours of life are self-limited by the way they conceive of things. in many cases people aren't even aware that they are limiting themselves. two of the great revolutions of twentieth century music are that (a) pitch doesn't have to be the most important parameter in music, and (b) even allowing music to continue to be pitch-centric, there are other ways to organize pitches beyond those taught in european classical theory books. certainly genres other than conceptual music have contributed to both of these revolutions, but conceptual music seems entirely dedicated to those ideas. insofar as it does that, i think it is valueable. the general public (and even most musicians) won't be directly impacted by conceptual music. but who can deny that innovation in music is exiting and interesting. the attention that twentieth-century music has given, for instance, to non-pitch parameters such as timbre and rhythm is a big part of what makes music interesting to me. having a keyboard with a million different sounds at one's disposal is a great joy. having programs like csound where one can design a new "instrument" at his leisure is liberating and exciting.
changing directions somewhat. as someone with a heavy background in science, math, and engineering, i may have a different view of what is beautiful than you or others here who have different backgrounds. i see beauty and elegance in symbolic things: grand-unified theories that integrate two or three of the physical forces, the golden ratio, chaos theory, automata (see wolfram), etc. i enjoy music that can turn beauty in one domain into (as i see it) beauty in a different domain. if one can, for instance, turn a well-played chess game into an interesting piece of music that captures the essence of the chess game while still presenting interesting aural artifacts, i find that very attractive. i know that a lot of people don't share these ideas of beauty that i have, but for those of us that do ... well, it's good stuff =)
i should append this by noting that i haven't ever come up with anything i would consider even vaguely conceptual art-like of my own. when i compose, it is largely in the style you are proposing, where one forgets about any rules he might know and just lets it happen. that being said, i'm glad that i know the ideas and rules that i do know, because i like using them to get out of ruts or generate new ideas when that black-box stream of creativity runs temporarily dry.