Hey Alex, whether or not holding a representation inside means there is “subjective experience” depends a lot on… well, your definitions of subjective and experience ;). And if you want to go so far as to say that subjective experience is a “feeling” we need yet more exploration of terminology.

My position on subjectivity is that it’s rather simple — if something is totally internal to a being, then it is subjective. If we put a computer in a box, and show it a random color, its sensors see it, determines that it’s #ff0000, that is subjective knowledge, only that computer knows it! But that is knowledge! not experience. The process of viewing the outside world and documenting that number, that’s an experience. The process of taking that number, and having other things happen in the machine, that’s experience as well. Experience comes as the computer uses that knowledge as part of higher-order abstractions. Certainly not the kind of thing I see in an electron or basic thermostat. However in an advanced device like a Nest we do see this kind of activity — the basic perceptual data is involved in higher-level constructs that affect behavior.

So the piece of paper with #FF0000 on it… maybe that’s knowledge, sure I guess that is a kind of knowledge right? But it isn’t experience if that knowledge is not being utilized as part of a large system.

If we want to go from experience to feeling… there’s a lot more to be said. Again, somewhat dependent on terminology… while we certainly can make machines that mimic human emotions, and do frequently with say videogame characters, I am more interested in the core feelings of machines that very much affect their behavior. I don’t see how machines could do what they do today without these core feelings. I like your word “incentive”…. nature may not have an incentive to make an electron feel (I tend not to think an electron has feelings) but there certainly is incentive for engineers to make computers have some feelings (but not too many). This however is a longer argument, I’m just finishing up my first book and a good chunk of it went into discussing the feelings of machines. If you PM me on Twitter @schkolne with your mailing address I should be able to get you an advance copy.