Share.

    16 Comments

    1. TheWesternMythos on

      Skimmed thru, have to double back later but a lot I liked at first glance!

      > The central argument describes three levels of regulation. “Proto-control” protects an organism’s integrity, such as by preserving a cell’s membrane and limiting destructive processes; the paper explicitly does not describe this as conscious. “Meta-control” manages conflicts among separate regulatory processes. “Settlement control” makes competing needs comparable and coordinates them into a shared course of action. 

      I have always thought of consciousness as more or less what “information processing”, past a certain complexity level ,feels like to the system doing the processing (yes lots of potential implications)

      > Intelligence and consciousness are not the same thing, and they did not evolve for the same reason. Intelligence is outward facing: it is for modeling the world, predicting, planning, solving problems. Consciousness evolved for something more internal: holding a fragile organism together when survival is on the line.

      Yes, one of the things this early AI age had taught me is a LOT of people have….less than ideal understanding of what intelligence is.

    2. Except AI doesn’t really grow by evolutionary processes. saying AI won’t become conscious because there’s no evolutionary pressure to share resources is like saying buildings won’t become tall because there’s no tree leaves to chew on.

    3. >The fear that we will accidentally scale our way into a suffering machine, and the engineering hype that digital minds are just around the corner, both rest on conflating intelligence with consciousness. If those are genuinely separate things, then consciousness is not something that sneaks up on us through sheer capability. It is a specific kind of organization we would have to deliberately build, and I would argue, one we have good reason not to. 

      Are there actually people who believe we will one day have the ability to give AI consciousness *but choose not to do it*?

    4. So… empathy? “But rather evolved as a distinct adaptation to help fragile organisms coordinate competing survival needs and manage self-maintenance”

    5. I appreciate the fact that this is theoretically testable, but I fail to see why something as elaborate as consciousness would need to evolve when cognition can resolve the conflict.

      At the same time, consciousness as a process seems shockingly inexpensive, given the creatures that can pass the mirror test, potentially even *ants*. Presumably their experience is far more primitive than our own, but the idea that it can exist at all is truly bizarre to me and suggests you can’t separate cognition and consciousness.

      I hate it, but I suspect the answer may just be that consciousness is just the emergent result of an information processing system that can internally reflect on itself (ie its outputs feed back into its inputs without extending into the outside world).

    6. v sauce had a pretty fitting video to that topic. can’t include links here.. but it’s called cognitive trade-off hypothesis on yt

    7. bigfatfurrytexan on

      Don’t we have models where the brain is a Bayesian reasoning machine? Would not the free energy principle fit in with that paradigm?

    8. iCliniq_official on

      This is such an interesting idea! We often link consciousness with being smart, but biology may be more complicated than that. The self-preservation angle is fascinating, although it’s still a theory, we’ll need much more evidence to know how consciousness actually evolved.

    9. I’ve often thought that we’re over thinking something that we’re seeing in such a wide range of organisms, which does seem to suggest it has survival value, or why else has it been preserved for so long?

      I would think that consciousness is the awareness of and the ability to respond to a given environment, which would also include a light-sensitive switch, perhaps less a bug than a feature of how adaptable and universal consciousness is.

    10. Vast-Masterpiece7913 on

      This is strong support for the concept behind the Orch OR theory of consciousness, in that it shows that consciousness, as predicted, is not an emergent property of brain complexity, but appropriate even for primitive organisms with the most basic nervous system.

    11. partymorphologist on

      They say

      *The fear that we will accidentally scale our way into a suffering machine, and the engineering hype that digital minds are just around the corner, both rest on conflating intelligence with consciousness.*

      But that’s not true either. I can define consciousness as the ability to observe and judge one’s own thought processes and adapt them. That’s not conflating or confusing, but could still lead to different outcomes.

      The article uses circular arguments here, would have to read the paper to confirm.

    12. I’ve long hypothesized that consciousness is not a unique trait to humans, that it’s not even unique to the “smart animals” (whales, elephants, parrots etc), but that it is very basal and that most animals have it, because a sense of self is necessary for basic stuff like calculating movement paths. It’s the other more extremely niche brain functions like abstract thought that are special to humans, not basic “sense of self”. Language isn’t even super unique to humans, it’s just extremely elaborated. We don’t have many brain traits that evolved JUST in hominids, it is better to assume most of the functions we have exist in other animals but to a different, less complex degree.

    13. This is a cognitive scientist who was a student of Dennett and is basically what I’d expect from that combo. I like that there are explicitly testable hypotheses, and it’s an interesting idea that is a bit different from the “mainstream” consciousness stuff in a way that isn’t kooky. I’ll be reading this one.

    14. Tricky_Tomorrow5325 on

      If anyone wants to see this taken to its most horrifying conclusion you should read blindsight

    15. >“The most important takeaway is that biology and evolution reframe the AI debate. The fear that we will accidentally scale our way into a suffering machine, and the engineering hype that digital minds are just around the corner, both rest on conflating intelligence with consciousness. If those are genuinely separate things, then consciousness is not something that sneaks up on us through sheer capability. It is a specific kind of organization we would have to deliberately build, and I would argue, one we have good reason not to. Getting that distinction right is how we choose our ethics rather than stumble into them.”

      >…“There is also a deeper question the paper raises but doesn’t settle. If we ever did build a system with genuine stakes in its own survival, we would have created something that can be harmed, a moral patient whose interests may not align with ours. That is a line worth understanding well before we approach it, not after.”

      A culture arises from the capacity to share subjective states of mind, and therefore achieve self-identification with independent but relate entities. That is also the basis for understanding the benefit and necessity of a moral order to minimize conflict and attune functioning. Bad faith actors seek to achieve benefits for “self” at the expense of “group”. Is there any reason to believe that AI entities will achieve greater social cohesion than their biological human progenitors?